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PREFACE 


This  report  documents  a  Federal  Aviation  Administration  operational 
evaluation  of  the  initial  group  of  four  air  traffic  control  (ATC) 
services  which  have  been  designed  for  implementation  on  the 
Mode  S  Data  Link  system.  The  report  is  organized  in  two  volumes. 

Volume  I  contains  the  main  body  of  the  report.  It  includes  a 
detailed  description  of  the  objectives  of  the  evaluation  and  of  the 
technical  approach  and  test  methods  that  were  used.  In  addition, 
the  primary  results  of  the  controller  and  aircrew  portions  of  the 
study,  conclusions,  and  recommendations  are  presented. 

Volume  II  contains  a  set  of  appendixes  to  the  report.  These 
appendixes  are  referenced  in  Volume  I  and  include  technical 
documentation  of  the  test  hardware  and  of  controller  and  pilot  Data 
Link  procedures.  The  appendixes  also  present  detailed  analyses  of 
the  data  which  were  collected  in  the  study.  These  appendixes  were 
the  basis  for  the  primary  results  sections  presented  in  Volume  I. 
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EXECUTIVE  SUMMARY 


INTRODUCTION . 

The  Federal  Aviation  Administration  (FAA)  is  pursuing  an  initiative 
to  develop  and  implement  a  Data  Link  system  intended  to  enhance 
communications  between  ground-based  air  traffic  control  (ATC)  and 
airborne  systems.  By  providing  digital  information  transfer  with 
the  ability  to  discretely  address  individual  aircraft.  Data  Link 
is  expected  to  relieve  frequency  congestion  on  existing  voice  radio 
channels  while  increasing  the  overall  safety  and  productivity  of 
the  ATC  system. 

In  order  to  insure  that  the  introduction  of  Data  Link  will  have  an 
optimal  positive  impact  on  ATC,  the  FAA  is  conducting  a  program  of 
research  to  guide  system  design  efforts,  evaluate  the  benefits  of 
Data  Link  to  ATC,  and  assess  its  effects  on  the  air  traffic 
controllers  and  aircrews  who  will  use  the  system.  Preliminary 
design  studies  completed  during  the  past  year  (Talotta,  et  al., 
1988,  1989)  defined  the  detailed  configurations  for  the  initial 
group  of  ATC  services  and  functions  scheduled  for  implementation 
on  Data  Link.  This  report  presents  the  results  of  the  first 
operational  evaluation  of  these  initial  capabilities  in  a  full 
scale,  ATC  simulation  manned  by  en  route  air  traffic  controllers. 
In  addition,  this  report  presents  the  findings  of  a  concurrent 
study  in  which  aircrews  flying  simulated  aircraft  in  the  ATC 
scenarios  evaluated  the  initial  services  and  a  preliminary  flight 
deck  display/control  device  fo*-  Data  Link  communications. 

OBJECTIVES. 

The  general  purpose  of  the  air  traffic  controller  substudy  was  to 
assess  the  impact  of  the  initial  package  of  Data  Link  ATC  services 
and  functions  in  the  context  of  a  high  fidelity  simulation  of  en 
route  ATC  activity.  The  specific  objectives  were  to  measure  the 
beneficial  effect  of  Data  Link  on  voice  radio  frequency  congestion 
and  to  determine  the  impact  on  ATC  performance  and  effectiveness, 
controller  workload,  and  controller  acceptance.  The  aircrew 
substudy  was  designed  to  evaluate  the  overall  effectiveness  of  Data 
Link  communications  in  aircraft  operations  both  for  the  ATC 
services  uplinked  by  the  subjects  in  the  controller  substudy,  and 
for  a  set  of  Data  Link  weather  services  available  by  direct  aircrew 
interaction  with  simulated  data  bases. 

DATA  LINK  SERVICES. 

Two  ATC  services  and  two  general  Data  Link  functions  were  tested 
in  the  controller  substudy.  The  ATC  services  were  altitude 
assignment  and  transfer  of  communication.  Data  Link  procedures  for 
both  of  these  services  were  designed  to  closely  parallel  current 
en  route  controller  procedures  with  the  addition  of  a  minimal 
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number  of  keyboard  actions.  The  status  of  Data  Link  transactions 
with  aircraft  were  displayed  in  a  Plan  View  Display  (PVD)  list  and 
in  the  Full  Data  Block  (FDB) .  Aircraft  equipage  and  uplink 
eligibility  were  indicated  by  graphic  characters  in  the  first  field 
of  the  FDB. 

The  Data  Link  ATC  functions  were  menu  text  and  free  text.  Menu 
text  permits  standard  or  frequently  required  ATC  messages  to  be 
stored  in  a  menu  displayed  on  the  PVD  and  selected  for  uplink  as 
required.  For  the  present  study,  menu  messages  were  restricted  to 
interim  altitude  assignments,  sometimes  combined  with  fix  crossing 
restrictions.  Free  text  provides  controllers  with  the  ability  to 
engage  in  unconstrained  ground-air  communications  using  typed 
keyboard  entries.  This  function  is  envisioned  primarily  as  a  back¬ 
up  communications  capability. 

In  addition  to  the  primary  ATC  services  delivered  by  the 
controllers,  the  aircrew  substudy  also  examined  a  group  of  weather 
services  which  the  pilots  could  request  by  direct  downlink  to  a 
data  base.  These  services  were  terminal  forecast,  winds/ 
temperatures  aloft,  surface  observation,  and  Pilot  Reports 
(PIREPS) .  The  aircrew  received  and  responded  to  Data  Link  messages 
on  a  touch  sensitive  display. 

APPROACH . 

The  evaluation  was  conducted  in  the  FAA  Technical  Center's  Data 
Link  Test  Bed.  Two  30-minute  test  scenarios  presented  controllers 
with  realistic  en  route  ATC  problems  involving  four  sectors  of 
adjacent  airspace.  A  majority  of  the  aircraft  in  the  scenarios 
were  simulated  radar  targets  controlled  by  simulation  operators. 
However,  two  of  the  targets  in  each  test  run  were  driven  by  high 
fidelity  simulators  representing  a  large  transport  aircraft  and  a 
small  General  Aviation  (GA)  aircraft.  The  airliner  was  manned  by 
professional  aircrews,  while  the  small  aircraft  was  flown  by  GA 
pilots. 

The  eight  en  route  controllers  participated  in  teams  of  four  in  14 
test  runs  in  which  Data  Link  equipage  levels  of  the  air  traffic 
ranged  from  0  (voice  radio  only)  to  20  and  70  percent.  In  one  of 
the  runs,  average  uplink  and  downlink  times  were  extended  to  assess 
the  impact  of  "worst  case"  Data  Link  communication  transmission 
delays.  Each  of  the  four  participating  aircrews  and  the  four  GA 
pilots  participated  in  two  Data  Link  runs  and  one  baseline,  voice- 
only  run. 

Controller  measures  included  records  of  voice  and  Data  Link  usage, 
ratings  of  controller  workload  and  service  design  acceptability, 
estimates  of  operational  effectiveness,  a  wrap-up  questionnaire, 
and  extensive  debriefing  to  elicit  required  Data  Link  design 
modifications.  Aircrew  measures  evaluated  perceptions  of  the 
effectiveness  of  the  services  and  the  usability  of  the  preliminary 
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flight  deck  Data  Link  Input/Output  (I/O)  device.  Times  required 
for  aircrew  to  detect  and  respond  to  Data  Link  and  voice  messages 
also  were  recorded. 

PRIMARY  RESULTS. 

The  controller  substudy  clearly  demonstrated  the  benefit  that  the 
initial  group  of  Data  Link  services  can  be  expected  to  have  on 
relieving  frequency  congestion  of  voice  radio  channels.  The 
availablity  of  these  functions  reduced  the  number  of  controller 
voice  transmissions  by  up  to  41  percent,  and  total  controller 
occupation  time  of  the  radio  frequencies  by  up  to  45  percent  (see 
figure  ES-1) .  Furthermore,  as  more  Data  Link  equipped  aircraft  were 
introduced,  there  was  an  apparent  increase  in  the  efficiency  of  the 
communication  process  as  requirements  for  repetition  of  voice 
messages  and  clarification  of  clearances  were  reduced. 

The  positive  impact  of  Data  Link  was  not  achieved  at  any  observed 
loss  in  ATC  performance  or  controller  capability.  No  critical  ATC 
incidents  were  noted  during  the  tests,  and  controller  workload 
estimates  were  statistically  identical  in  the  voice  and  Data  Link 
conditions  (figure  ES-1) .  The  results  also  showed  that  Data  Link 
transmission  delays  failed  to  have  a  negative  effect  on  controller 
activities.  All  of  the  specific  designs  for  the  Data  Link  services 
were  rated  as  operationally  effective  and  acceptable  to 
controllers. 

The  results  of  the  aircrew  substudy  indicated  that  initial  pilot 
perceptions  of  Data  Link  were  positive  and  that  its  implementation 
would  enhance  the  quality  of  air-ground  communications.  Additional 
findings  indicated  that  the  average  total  times  between  receipt  of 
a  message  in  the  aircraft  and  the  crew  response  to  ATC  ranged  from 
9.8  seconds  in  the  voice  conditions  to  10.9  seconds  in  the  Data 
Link  conditions. 

The  workload  associated  with  Data  Link  operations  was  considered 
acceptable  to  pilots.  Furthermore,  while  the  aircrew  subjects 
indicated  further  development  of  display  formats  and  crew 
coordination  procedures  will  be  required,  they  felt  that  persistent 
Data  Link  displays  will  reduce  demands  on  pilot  memory  and  that 
Data  Link  will  be  capable  of  replacing  most  functions  currently 
served  by  voice  radio  in  en  route  environments. 

RECOMMENDATIONS . 

Based  on  the  results  outlined  above,  it  is  recommended  that  the 
initial  group  of  Data  Link  ATC  services  be  incorporated  as 
modifications  to  current  en  route  ATC  software  and  hardware,  and 
that  they  be  subjected  to  operational  test  and  evaluation  (OT&E) . 
Minor  outstanding  design  issues  and  potential  enhancements  revealed 
by  the  present  study  should  be  examined  by  the  Air  Traffic  Data 
Link  Validation  Team  (ATDLVT)  and  resolved.  It  is  further 
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recommended  that,  because  of  the  success  of  the  operational 
evaluation  and  the  positive  impressions  expressed  by  participating 
controllers  and  pilots,  research  and  development  work  be  expedited 
to  extend  Data  Link  applications  to  the  full  range  of  en  route  and 
terminal  ATC  services. 

Because  the  development  of  refined  flight  deck  interfaces  has 
lagged  behind  corresponding  ground  controller  work,  it  is 
recommended  that  additional  studies  be  conducted  to  facilitate  the 
integration  of  Data  Link  with  the  aircrew  task  environment. 

Finally,  in  order  to  identify  mutually  acceptable  rules  and 
procedures  which  will  govern  pilot  and  controller  communications 
using  Data  Link,  it  is  recommended  that  a  mechanism  be  created  to 
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INTRODUCTION 


A  total  of  six  operational  evaluation  scenarios  were  used  in  the 
evaluation  of  the  Phase  I  Data  Link  services  functions.  Two  of  the 
scenarios  were  used  to  train  the  subject  controllers  and  the  other 
four  were  used  to  conduct  the  evaluation  and  collect  data.  The 
scenarios  are  the  Operational  Evaluation  Training  Scenarios  (one 
with  70  percent  Data  Link-equipage  and  one  with  100  percent  Data 
Link-equipage) ,  and  the  Operational  Evaluation  Scenarios  (two  with 
20  percent  Data  Link-equipage  and  two  with  70  percent  Data 
Link-equipage) .  All  of  the  scenarios  were  designed  with  the 
Universal  Data  Set  (UDS)  simulated  airspace  and  run  for  35  minutes. 
This  appendix  describes  these  scenarios,  the  relevant  Menu  Text 
messages  used,  and  the  En  Route  Controller  Data  Link  chart  that 
depicts  the  controller  Data  Link  input  message  procedures. 

DESCRIPTION  OF  THE  OPERATIONAL  EVALUATION  TRAINING  SCENARIO 

The  Operational  Evaluation  Training  Scenario  simulates  a  low  volume 
en  route  air  traffic  environment.  A  description  of  the  training 
scenarios  with  70  and  100  percent  Data  Link-equipage  are  provided 
below. 

OPERATIONAL  EVALUATION  TRAINING  SCENARIO  WITH  70  PERCENT 

DATA  LINK-EQUIPAGE 

This  training  scenario  simulates  a  traffic  flow  that  forms  a 
circular  routing  between  two  airports.  As  shown  in  figure  A-l,  two 
low  altitude  sectors  (03  and  17)  and  two  high  altitude  sectors  (07 
and  10) ,  are  represented.  Figure  A-2  outlines  the  routes  and  the 
number  of  aircraft  and  Data  Link-equipped  aircraft  in  the  scenario. 
The  total  number  of  aircraft  (45)  includes  six  aircraft  (not  shown 
in  the  figure  A-2)  that  appear  at  the  start  of  the  run. 

At  the  start  of  the  scenario,  six  aircraft  appear  in  flight  in  the 
high  altitude  sectors  (three  aircraft  per  sector) .  This  provides 
an  initial  air  traffic  control  (ATC)  situation  for  the  high 
altitude  controllers.  The  departing  aircraft  appear  simultaneously 
in  sectors  03  and  17  at  the  two  airports. 

The  aircraft  starting  in  sector  03  appear  at  12,000  feet  on  Rt.  1 
(see  figure  A-2) .  This  route  contains  nine  aircraft  (six  are  Data 
Link-equipped)  departing  the  western  airport  (Philadelphia 
International)  at  fix  point  D3B.  As  the  aircraft  climb,  they  are 
given  a  clearance  via  a  Menu  Text  message  or  radio  to  climb  and 
maintain  flight  level  230.  The  westbound  overflight  route  contains 
nine  aircraft  (five  are  Data  Link-equipped).  Of  the  nine  aircraft, 
five  are  low  altitude  overflights  that  are  designed  to  conflict 
with  the  departures  in  sector  03  (three  aircraft  are  Data  Link- 
equipped)  .  After  the  clearance  is  received,  the  aircraft  are 
handed  off  from  sector  03  to  high  altitude  sector  07. 
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In  sector  07,  the  aircraft  are  given  a  clearance  via  a  Menu  Text 
message  or  radio  to  cross  KMART  and  maintain  flight  level  250.  The 
westbound  overflight  route  also  contains  high  altitude  overflights 
that  are  designed  to  impact  the  traffic  in  sector  07.  This 
overflight  route  contains  four  high  altitude  overflights  (two  are 
Data  Link-equipped) .  The  aircraft  in  sector  07  are  handed  off  to 
high  altitude  sector  10.  The  aircraft  are  given  a  clearance  via  a 
Menu  Text  message  or  radio  to  cross  BOIDS  and  maintain  flight  level 
240.  The  aircraft  in  this  sector  are  designed  to  be  in  conflict 
with  the  overflights  starting  at  fix  point  F2E  (see  figure  A-2) . 
This  overflight  route  contains  12  eastbound  overflights  (7  are  Data 
Link-equipped).  Of  the  12  overflights,  4  are  high  altitude  (3  are 
Data  Link-equipped) .  The  aircraft  in  sector  10  are  handed  off  to 
low-altitude  sector  17. 

A  clearance  via  a  Menu  Text  message  or  radio  to  cross  HOLTS  and 
maintain  12,000  feet  is  given  to  the  aircraft  in  sector  17.  The 
low  altitude  overflights,  of  which  four  are  Data  Link-equipped,  are 
designed  to  conflict  with  the  aircraft  in  this  sector.  As  the 
aircraft  approach  Atlantic  City  International  Airport  (G2E) ,  they 
are  terminated  from  the  scenario. 

The  aircraft  starting  in  sector  17  also  appear  at  12,000  feet  on 
Rt.  2  (see  figure  A-2) .  This  route  contains  nine  aircraft  (six  are 
Data  Link-equipped)  departing  the  eastern  airport  (Atlantic  City 
International)  at  fix  point  G2E.  As  each  aircraft  appears  in  the 
sector  they  are  given  a  clearance  via  a  Menu  Text  message  or  radio 
to  climb  and  maintain  flight  level  230,  and  then  are  handed  off  to 
high-altitude  sector  10.  These  aircraft  are  given  a  clearance  via 
a  Menu  Text  message  or  radio  to  cross  RHOME  and  maintain  flight 
level  260.  The  high  altitude  overflights,  described  in  the  above 
paragraph,  are  designed  to  conflict  with  the  aircraft  in  this 
sector.  The  aircraft,  after  receiving  the  clearance,  are  handed  off 
to  high  altitude  sector  07. 

When  the  aircraft  enter  sector  07,  they  are  given  a  clearance  via 
a  Menu  Text  message  or  radio  to  cross  GLADD  and  maintain  flight 
level  240.  The  aircraft  are  impacted  by  the  high  altitude 
overflights  on  the  eastbound  overflight  route  that  is  described  in 
the  above  paragraph.  The  aircraft  in  sector  07  are  handed  off  to 
low  altitude  sector  03.  A  clearance  via  a  Menu  Text  message  or 
radio  to  cross  BATON  and  maintain  12,000  feet  is  given  to  the 
aircraft  in  sector  03.  The  aircraft  are  terminated  from  the 
scenario  at  fix  point  D3B  (Philadelphia  International  Airport) . 

The  Menu  Text  messages  for  each  of  the  four  sectors  in  this 
scenario  are  shown  in  figure  A-3.  These  messages  appear  on  the 
sector  position  Plan  View  Displays  (PVD's)  at  the  beginning  of  the 
run.  The  controllers  send  the  appropriate  Menu  Text  message  to  the 
aircraft  as  the  situation  warrants.  The  messages  with  the  leading 
plus  sign,  when  sent  to  the  aircraft,  update  the  altitude  field  in 
the  aircraft's  full  data  block  (FDB)  on  the  PVD. 
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FIGURE  A-3 .  MENU  TEXT  DATA  LINK  OPERATIONAL  EVALUATION  TRAINING  SCENARIO 


OPERATIONAL  EVALUATION  TRAINING  SCENARIO  WITH  100  PERCENT 

DATA  LINK-EOUIPAGE 

This  scenario  (see  figure  A-4)  is  identical  to  the  training 
scenario  with  70  percent  Data  Link  -  equipage,  except  that  no 
overflight  routes  are  used  and  the  total  of  24  aircraft  simulated 
are  all  Data  Link-equipped.  ' 

DESCRIPTION  OF  THE  OPERATIONAL  EVALUATION  SCENARIO  #1 

The  Operational  Evaluation  Scenario  #1  simulates  a  medium  volume 
en  route  air  traffic  environment.  Two  scenarios  (one  with  70 
percent  Data  Link-equipage  and  one  with  20  percent  Data 
Link-equipage)  are  described  below. 

OPERATIONAL  EVALUATION  SCENARIO  #1  WITH  70  PERCENT 

DATA  LINK-EOUIPAGE 

This  scenario  simulates  a  traffic  flow  that  forms  a  circular 
routing  between  two  airports  as  shown  in  figure  A-5.  There  are  two 
low  altitude  sectors  (03  and  17)  and  two  high  altitude  sectors  (07 
and  10) .  The  routes  and  the  number  of  aircraft  and  Data 
Link-equipped  aircraft  are  depicted  in  figure  A-6.  This  scenario 
has  a  total  of  63  aircraft.  In  addition,  two  aircraft  simulators 
were  used  in  this  scenario:  a  Cessna  421  general  aviation  flight 
simulator  (GAT  II)  located  at  the  FAA  Technical  Center  and  a  Boeing 
727  flight  simulator  (NASA/AMES) . 

When  the  scenario  begins,  six  aircraft  appear  in  flight  in  the 
high  altitude  sectors  (three  aircraft  per  sector) .  This  provides 
an  initial  ATC  situation  for  the  high  altitude  controllers.  The 
departing  aircraft  begin  simultaneously  in  both  low  altitude 
sectors  (03  and  17)  at  each  airport.  When  the  aircraft  appear  in 
the  low  altitude  sector  03,  they  are  at  12,000  feet  on  Rt.  1  (see 
figure  A-6)  departing  the  western  airport  (Philadelphia 

International)  at  fix  point  D3B.  This  route  contains  nine  aircraft 
(six  are  Data  Link-equipped) .  The  aircraft  are  given  a  clearance 
via  a  Menu  Text  message  or  radio  to  climb  and  maintain  flight  level 
230.  Within  sector  03,  there  are  two  overflight  routes,  Rt.  3  and 
Rt.  4  (see  figure  A-6) .  These  routes  contain  the  same  fix  points  ^ 

but  the  traffic  flows  in  opposite  directions.  Of  the  12  aircraft, 
six  aircraft  (four  are  Data  Link-equipped)  are  low  altitude 
overflights  that  are  designed  to  conflict  with  the  departures  in 
sector  03.  Upon  receipt  of  the  clearance,  the  aircraft  are  handed 
off  from  sector  03  to  high-altitude  sector  07. 

When  the  aircraft  are  accepted  into  sector  07,  they  are  given  a 
clearance  via  a  Menu  Text  message  or  radio  to  cross  KMART  and 
maintain  flight  level  250.  The  bidirectional  overflight  route 
described  above  also  contains  six  high  altitude  overflights  (four 
are  Data  Link-equipped)  that  are  designed  to  impact  the  traffic  in 
sector  07.  In  addition,  there  is  another  high  altitude  overflight 
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route  (High  Altitude  Additions)  that  contains  three  overflights 
(two  are  Data  Link-equipped) .  High  altitude  sector  10  accepts 
aircraft  transferred  from  sector  07. 

The  sector  10  controller  gives  a  clearance  via  a  Menu  Text  message 
or  radio  to  cross  BOIDS  and  maintain  flight  level  240  after  the 
aircraft  enter  the  sector.  The  two  overflight  routes,  Rts.  5  and  * 

6  (see  figure  A-6) ,  create  conflicting  traffic  in  this  sector. 

These  routes  contain  the  same  fix  points,  but  the  traffic  flows  in 
two  directions.  These  routes  contain  12  high  altitude  overflights  1 

(9  are  Data  Link-equipped) .  Additional  high  altitude  overflights 
are  presented  on  the  route  labeled  High  Altitude  Additions  (see 
figure  A-6)  that  contain  six  aircraft  of  which  five  are  Data 
Link-equipped.  The  aircraft  in  sector  10  are  handed  off  to  low 
altitude  sector  17. 

After  the  aircraft  enter  sector  17,  the  controller  gives  a 
clearance  via  a  Menu  Text  message  or  radio  to  cross  HOLTS  and 
maintain  12,000  feet.  The  aircraft  in  this  sector  are  designed  to 
conflict  with  low  altitude  overflights  on  the  overflight  route 
labeled  Low  Altitude  Additions.  There  are  six  low  altitude 
overflights  (four  are  Data  Link-equipped).  The  aircraft  are 
terminated  from  the  scenario  at  fix  point  H2A  (Atlantic  City 
International  Airport) . 

The  aircraft  moving  westbound  appear  in  sector  17  at  12,000  feet, 
on  Rt.  2  (see  figure  A-6)  departing  the  eastern  airport  (Atlantic 
City  International)  at  fix  point  H2A.  This  route  contains  nine 
aircraft  (six  are  Data  Link-equipped) .  The  aircraft  are  given  a 
clearance  via  Menu  Text  message  or  radio  to  climb  and  maintain 
flight  level  230.  The  aircraft  are  impacted  by  low  altitude 
overflights  on  the  Low  Altitude  Additions  route  described  in  the 
above  paragraph.  After  the  clearance  is  given,  the  aircraft  are 
handed  off  to  high  altitude  sector  10.  When  the  aircraft  enter 
sector  10,  they  are  given  a  clearance  via  a  Menu  Text  message  or 
radio  to  cross  RHOME  and  maintain  flight  level  260.  These  aircraft 
are  impacted  by  the  high  altitude  overflight  route  described  in  the 
above  paragraph.  The  controller  in  sector  10  hands  off  aircraft  to 
high-altitude  sector  07. 

When  the  aircraft  are  accepted  in  sector  07,  a  clearance  is  given 
via  a  Menu  Text  message  or  radio  to  cross  GLADD  and  maintain  flight 
level  240.  The  high  altitude  overflights  that  are  described  in  the 
above  paragraph  conflict  with  the  aircraft  in  this  sector.  The 
aircraft  in  sector  07  are  handed  off  to  low  altitude  sector  03.  A 
clearance  via  a  Menu  Text  message  or  radio  to  cross  BATON  and 
maintain  12,000  feet  is  given  to  the  aircraft  in  sector  03.  The 
aircraft  are  terminated  from  the  scenario  as  they  approach 
Philadelphia  International  Airport. 

The  NASA/AMES  simulator  represents  a  departure  traveling  along 
Rt.  1  (see  figure  A-6)  and  appears  to  the  controller  no  different 
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than  the  other  aircraft  in  the  scenario.  Also,  the  GAT  II  simulator 
was  assigned  a  special  route  as  depicted  in  figure  A-6.  This 
aircraft  starts  at  8,000  feet  in  low  altitude  sector  17,  climbs  to 
12,000  feet,  and  then  is  handed  off  to  low  altitude  sector  03.  The 
GAT  II  completes  its  route  in  sector  03  at  an  airport  defined  at 
fix  oint  3ETF. 

In  figure  A-7,  the  Menu  Text  messages  are  shown  for  each  of  the 
four  sectors  in  this  scenario.  When  the  run  begins,  these  messages 
are  displayed  on  the  sector  PVD's.  The  appropriate  message  is 
sent,  when  the  controller  takes  an  action,  to  the  aircraft  as  the 
situation  warrants.  The  messages  with  the  leading  plus  sign,  when 
sent  to  the  aircraft,  update  the  altitude  field  in  the  aircraft's 
full  data  block  on  the  PVD. 

OPERATIONAL  EVALUATION  SCENARIO  #1  WITH  20  PERCENT 

DATA  LINK-EQUIPAGE 

This  scenario  consists  of  a  total  of  63  aircraft  of  which  19  are 
Data  Link-equipped,  as  shown  in  figure  A-8.  The  number  of  Data 
Link-equipped  aircraft  is  the  only  difference  between  the  two 
versions  of  Operational  Evaluation  Scenario  #1. 

DESCRIPTION  OF  OPERATIONAL  EVALUATION  SCENARIO  #2 

The  Operational  Evaluation  Scenario  #2  simulates  a  realistic,  high 
volume  en  route  air  traffic  environment  sample  of  arrival  and 
departure  sectors.  The  information  below  describes  the  scenarios 
with  70  and  20  percent  Data  Link-equipage. 

OPERATIONAL  EVALUATION  SCENARIO  #2  WITH  70  PERCENT 

DATA  LINK-EQUIPAGE 

This  scenario  represents  two  separate  traffic  flows  that  operate 
independently  of  each  other.  The  two  traffic  flows  represent 
arrival  and  departure  routes  in  an  en  route  center.  In  each  of  the 
two  traffic  flows  there  is  one  low  altitude  sector  and  one  high 
altitude  sector  as  shown  in  figure  A-9.  Of  the  78  aircraft,  4  (not 
shown  in  figure  A-10)  appear  on  the  arrival  routes  at  the  start  of 
the  run.  In  addition,  two  aircraft  simulators  were  used  in  this 
scenario:  a  Cessna  421  GAT  II  located  at  the  FAA  Technical  Center 
and  a  Boeing  727  flight  simulator  (NASA/AMES) . 

The  first  of  the  two  traffic  flows  in  the  scenario  simulate 
departures  from  an  airport  (upper  portion  of  figure  A-10)  that  fly 
through  low  altitude  departure  sector  03  and  high  altitude 
departure  sector  10.  The  aircraft  begin  in  sector  03  at  12,000  feet 
along  one  of  the  two  departure  routes,  Rt.  1  or  Rt.  2  (see  figure 
A-10)  .  Each  departure  route  contains  12  aircraft  (10  in  each  route 
are  Data  Link-equipped)  departing  the  western  airport  (Philadelphia 
International)  at  fix  point  D3B .  The  aircraft  are  given  a  clearance 
via  a  Menu  Text  message  or  radio  to  climb  and  maintain  flight  level 
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FIGURE  A- 7 .  MENU  TEXT  DATA  LINK  OPERATIONAL  EVALUATION  SCENARIO  #1 


OVERFLIGHTS 
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FIGURE  A-9 .  AIRSPACE  SECT0RIZATI0N  DATA  LINK  OPERATIONAL 
EVALUATION  SCENARIO  #2 
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FIGURE  A-10.  AIRCRAFT  ROUTING  DATA  LINK  OPERATIONAL  EVALUATION  SCENARIO  //2 
(70%  DATA  LINK-EQUIPAGE) 


230.  Within  sector  03,  there  is  one  overflight  route,  Rt.  3  (see 
figure  A-10) ,  containing  seven  aircraft  (three  are  Data 
Link-equipped)  that  are  designed  to  conflict  with  the  departures. 
The  sector  03  controller  hands  off  aircraft  to  high  altitude  sector 
10.  The  aircraft  in  this  sector  conflict  with  aircraft  on  the  two 
high  altitude  overflight  routes.  Each  of  the  overflight  routes  in 
sector  10  contains  seven  aircraft  (three  in  each  route  are  Data 
Link-equipped) .  This  portion  of  the  scenario  contains  45  aircraft 
(29  are  Data  Link-equipped) . 

The  second  of  the  two  traffic  flows  simulates  aircraft  arriving  at 
an  airport  (lower  portion  of  figure  A-10)  that  fly  through  high 
altitude  arrival  sector  07  and  low  altitude  arrival  sector  17.  At 
the  start  of  this  scenario,  four  aircraft  are  already  en  route. 
This  provides  an  initial  ATC  situation  in  this  portion  of  the 
scenario.  The  remaining  aircraft  begin  in  high  altitude  arrival 
sector  07  along  one  of  the  two  routes,  Rt.  6  and  Rt.  7  (see  figure 
A-10) .  Rt.  6  contains  13  aircraft  (11  are  Data  Link-equipped)  and 
Rt.  7  contains  9  aircraft  (8  are  Data  Link-equipped) .  The  aircraft 
along  Rts.  6  and  7  are  given  a  clearance  via  a  Menu  Text  message 
or  radio  to  descend  and  maintain  to  flight  level  240  and/or  cross 
HICOE  and  maintain  flight  level  240.  As  the  aircraft  are  separated, 
they  are  merged  into  one  stream  of  traffic  before  they  are  handed 
off  to  low  altitude  arrival  sector  17.  When  sector  17  receives 
control  of  the  aircraft,  a  clearance  via  a  Menu  Text  message  or 
radio  is  given  to  descend  and  maintain  an  altitude  of  12,000  feet 
and/or  cross  FLATO  and  maintain  an  altitude  of  12,000  feet.  One 
low  altitude  overflight  route,  Rt.  10,  that  contains  seven  aircraft 
(four  are  Data  Link-equipped)  interfere  with  the  stream  of 
arrivals.  All  of  the  arrivals  are  terminated  at  the  eastern  airport 
(Atlantic  City  International)  at  fix  point  G4F .  This  portion  of  the 
scenario  contains  29  aircraft  (23  are  Data  Link-equipped) . 

The  NASA/AMES  simulator  was  a  departure  traveling  along  Rt.  2  (see 
figure  A-10)  and  appears  to  the  controller  as  one  of  the  Boeing  727 
aircraft.  This  aircraft  participated  in  the  scenario  in  the  same 
manner  as  the  other  aircraft  on  this  route.  The  GAT  II  simulator 
was  assigned  a  special  route  to  travel  that  is  depicted  in  figure 
A-10.  This  aircraft  starts  at  9,000  feet  in  low  altitude  departure 
sector  03,  climbs  to  10,000  feet,  and  then  handed  off  to  low 
altitude  arrival  sector  17.  The  GAT  II  completes  its  route  in 
sector  17  at  an  airport  defined  at  fix  point  F3G. 

The  Menu  Text  messages,  depicted  in  figure  A-ll,  defined  for  each 
of  the  four  sectors  in  this  scenario,  appear  on  the  sector  PVD's 
at  the  beginning  of  the  run.  As  the  situation  warrants,  the 
appropriate  Menu  Text  message  is  sent,  upon  a  computer  input 
action,  to  the  aircraft.  The  messages  with  the  leading  plus  sign 
when  sent  to  the  aircraft  update  the  altitude  field  in  the 
aircraft's  FDB  on  the  PVD. 


A-16 


w 

S5S|§a 

ooq9<h 

SSsSi^ 

Txxgoaw 

^SSs§ol 

Prt'H'-<  UUfi 
•j  +  +  CtQ 

jSkfflUQpiN 


W 

Q 

gw 

do 

gg 

s§ 

-Is 

^Ow 

hWW 

JtfQ 

Tj  .  . 

£c*Sl 


w 

Q 

DW 

|g^| 

ss|5 

33KEQ 

■dd|g 

2  fO  rn  OUp 
^c^rjgQgq 

J  +&Q 
yj<CQC*Si 


w 

z§zzgm 

QScug^p 

§|O0SS 

asa**^ 

’gQxXwg 

:ooogSr 

)  Tj-  O  w 
;  <N  <s  evj  ^  UJ  UJ 

;  +  +  c*Q 

3<cquQq£N 


A-17 


FIGURE  A- 1 1 .  MENU  TEXT  DATA  LINK  OPERATIONAL  EVALUATION  SCENARIO  //2 


This  scenario  consists  of  78  aicraft  of  which  18  are  Data 
Link-equipped  as  shown  in  figure  A-12.  The  number  of  Data  Link- 
equipped  aircraft  is  the  only  difference  between  the  two  versions 
of  Operational  Evaluation  Scenario  #2. 

EN  ROUTE  RADAR  CONTROLLER  DATA  LINK  CHART 

These  charts  (see  figure  A-13)  are  quick  reference  guides  on  the 
Data  Link  services/functions  message  input  format.  The  Data  Link 
services/functions  listed  on  the  charts  are  those  included  in  the 
test  bed.  The  first  two  charts  provide  procedures  for  entering  Data 
Link  messages;  the  last  two  charts  provide  additional  information 
on  the  Data  Link  symbology  and  Data  Link  status  list. 
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FIGURE  A-12.  AIRCRAFT  ROUTING  DATA  LINK  OPERATIONAL  EVALUATION  SCENARIO  it  2 
( 20 7,  DATA  LINK-EQUIPAGE) 
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FIGURE  A-13.  EN  ROUTE  RADAR  CONTROLLER  DATA  LINK  CHART  (SHEET  2  OF  A) 
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FIGURE  A- 13.  EN  ROUTE  RADAR  CONTROLLER  DATA  LINK  CHART  (SHEET  3  OF  4) 
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APPENDIX  B 

MEASURES  OF  CONTROLLER  WORKLOAD,  DESIGN  PREFERENCE, 
AND  OPERATIONAL  EFFECTIVENESS 
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INTRODUCTION 


This  appendix  presents  the  detailed  results  of  the  operational 
evaluation  study  that  were  obtained  from  controllers  during  the 
simulation  runs  in  the  Data  Link  test  bed.  These  results  were 
derived  from  overall  workload  ratings  taken  after  each  test 
condition,  and  from  ratings  of  operational 
effectiveness/suitability  and  controller  acceptance/preference 
collected  immediately  after  each  subject  had  completed  the  full 
series  of  seven  test  runs.  In  addition,  written  comments  elicited 
from  the  controllers  during  the  test  bed  sessions  are  summarized 
here. 


WORKLOAD  MEASUREMENT 

SUBJECTIVE  WORKLOAD  ASSESSMENT  TECHNIQUE  (SWAT)  SCALE  DEVELOPMENT. 

Controller  workload  was  assessed  by  the  test  subjects  immediately 
following  each  simulation  run  using  a  subjective  rating  method 
known  as  the  SWAT.  SWAT  was  developed  in  the  early  1980's  by  U.S. 
Air  Force  researchers  in  order  to  provide  a  standard  rating  method 
that  results  in  quantitative,  interval  level  estimates  of  the 
workload  experienced  by  operators  during  simulation  and  operational 
tests  (Reid  and  Nygren,  1988)  .  SWAT  was  used  successfully  for 
evaluation  of  air  traffic  controller  workload  in  both  of  the 
preliminary  Data  Link  Mini  Studies  (Talotta,  et  al.,  1988,  1989). 

SWAT  consists  of  a  rating  scale  which  requires  the  subject  to  judge 
the  workload  experienced  during  a  period  of  task  performance  as  a 
combination  of  ratings  on  three,  3-point  scales  representing  the 
dimensions  of  time  load,  mental  effort  load,  and  psychological 
stress.  The  SWAT  scale  is  shown  in  figure  B-l. 

In  order  to  generate  quantitative  data  using  this  scale,  subjects 
provide  information  on  how  time,  effort,  and  stress  combine  to 
produce  their  individual  concepts  of  mental  workload.  This  scale 
development  exercise  is  accomplished  prior  to  actual  data 
collection  by  asking  subjects  to  sort  a  set  of  27  cards  on  which 
are  printed  all  possible  combinations  of  the  descriptors  on  the 
time,  effort,  and  stress  scales.  Subjects  sort  the  cards  to 
produce  an  ordering  that  represents  situations  ranging  from  low  to 
high  workload.  The  sorts  are  then  subjected  to  computerized 
conjoint  analysis  which  determines  the  combinatory  rule  governing 
each  subject's  sequence.  An  iterative  routine  is  then  applied  to 
generate  an  interval  scale  value  for  each  of  the  possible  time, 
effort  and  stress  ratings  which  preserves  the  original  ordering  and 
fits  the  identified  rule.  The  result  of  this  process  is  a  look¬ 
up  table  for  each  subject,  or  group  of  similar  subjects,  that 
converts  the  discrete  time,  effort,  and  stress  ratings  to  an 
uveidil  unitary  scale  with  values  ranging  from  0  to  100. 
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TIME  LOAD 


1.  Of  tan  have  apara  tima.  Intaruptiona  or  ovarlap 

among  activltlaa  occur  infrequently  or  not  at  all. 

2.  Occasionally  hava  apara  time.  Intaruptiona  or 

ovarlap  among  activltlaa  occur  frequently. 

3.  Almost  never  have  apara  time.  Intaruptiona  or 
overlap  among  activities  are  very  frequent  or 
occur  all  the  time. 

MENTAL  EFFORT  LOAD 

1.  Vary  little  conacioua  mental  effort  or 

concentration  required.  Activity  la  almost 
automatic,  requiring  little  or  no  attention. 

2.  Moderate  conacioua  mental  effort  or  concentration 

required.  Complexity  of  activity  la  moderately 
high  due  to  uncertainty,  unpredictability,  or 
unfamlllarlty.  Considerable  attention  le 
required. 

3.  Extensive  mental  effort  and  concentration 

are  necessary.  Very  complex  activity 
requiring  total  attention. 

PSYCHOLOGICAL  8TRES8  LOAD 

1.  Little  confusion,  risk,  frustration  or  anxiety 

and  can  eaaily  be  accomodated. 

2.  Moderate  stress  due  to  confusion,  frustration  or 

anxiety.  Noticeably  adds  to  workload. 

Significant  compensation  is  required  to  maintain 
adequate  performance. 

3.  High  to  very  Intense  stress  due  to  confusion, 

frustration  or  anxiety.  High  to  extreme 
determination  and  self-control  required. 


FIGURE  B-l.  SWAT  RATING  SCALE 
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Prior  to  participation  in  the  test  bed  sessions,  all  eight  subjects 
completed  the  SWAT  card  sort  task  described  above.  The 
instructions  provided  to  subjects  for  the  card  sort  and  for  use  of 
the  SWAT  scale  are  included  as  the  final  section  of  this  appendix. 

A  Kendall's  Coefficient  of  Concordance  computed  on  the  card  sorts 
produced  by  the  eight  subjects  yielded  an  index  of  association 
among  the  card  orderings  of  0.79.  In  the  SWAT  methodology,  this 
level  of  agreement  is  considered  acceptable  for  computation  of  a 
group  scale.  To  evaluate  the  "goodness  of  fit"  of  the  average 
group  sort  data  to  an  additive  model,  the  required  axiom  tests  of 
conjoint  measurement  were  applied.  These  tests  resulted  in  0 
violations  out  of  324  tests  for  the  independence  axiom,  0 
violations  out  of  8  tests  for  the  double  cancellation  axiom,  and 
7  violations  out  of  324  tests  for  the  joint  independence  axiom. 
Since  this  finding  is  well  below  the  SWAT  criterion  of  20  total 
violations  for  rejection  of  the  model,  a  group  scale  was  computed 
for  interpretation  of  each  of  the  27  possible  SWAT  time,  effort, 
and  stress  ratings.  This  numerical  scale  assigned  an  approximate 
importance  of  37.37  percent  to  the  time  factor,  21.67  percent  to 
the  effort  factor,  and  40.96  percent  to  the  stress  factor.  All 
subsequent  analyses  of  SWAT  ratings  obtained  from  the  subjects  in 
the  test  bed  runs  were  performed  on  the  corresponding  values 
obtained  from  this  unitary  scale.  Table  B-l  presents  the  card 
order  data  for  each  subject  as  well  as  the  derived  scale  values 
assigned  to  each  possible  rating. 

SWAT  WORKLOAD  RESULTS. 

All  subjects  completed  a  SWAT  rating  immediately  following  each  of 
the  seven  test  runs.  The  primary  statistical  analysis  performed 
on  the  SWAT  scores  was  based  on  the  data  from  the  basic  test  runs 
in  which  the  subjects  controlled  traffic  in  a  baseline  (no  Data 
Link)  condition,  and  under  20  percent  and  70  percent  mixes  of 
aircraft  Data  Link  equipage.  These  conditions  were  repeated  for 
two  different  ATC  scenarios,  yielding  six  test  runs  for  each 
subject.  Subjects  were  tested  in  two  groups  of  four  subjects  each. 
The  two  groups  experienced  the  baseline  and  Data  Link  conditions 
in  different  sequences  in  order  to  reduce  rating  bias  and  to  permit 
inspection  of  the  effects  of  scenario  familiarization. 

A  three-factor  ANOVA  (Group  x  Scenario  x  Equipage)  with  repeated 
measures  on  the  final  two  factors  was  performed  on  the  data  from 
the  basic  test  runs.  This  analysis  tested  the  significance  of  each 
of  these  main  effects  as  well  as  any  interactions  among  the 
factors.  Figure  B-2  shows  the  mean  SWAT  scores  produced  by  the 
combined  groups  as  a  function  of  Data  Link  equipage  level  and  test 
scenario. 

The  results  of  the  analysis  revealed  no  significant  effect  of  any 
of  the  three  primary  factors  (F<1) .  That  is,  SWAT  ratings  did  not 
differ  significantly  between  groups,  scenarios,  or  among  the 
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TABLE  B-l .  SWAT  CARD  SORTS  AND  GROUP  SCALE  CONTROLLER  SURVEY 
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Note:  TES  =  Time  Load,  Mental  Effort,  and  Psychological  Stress  Load 
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B-2.  CONTROLLER  WORKLOAD  IN  VOICE  ONLY  AND  DATA  LINK 
CONDITIONS 


B-5 


baseline  and  Data  Link  test  conditions.  In  addition,  no  two-way 
interactions  among  the  factors  were  significant.  However,  the 
three-way  interaction  among  groups,  scenarios,  and  equipage  levels 
was  significant  (F (2 , 12 ) =3 . 89 ,  p<.05). 

This  complex  interaction  effect  can  be  interpreted  by  inspecting 
the  effect  of  testing  sequence  within  the  two  test  scenarios.  In 
designing  the  operational  evaluation  study,  it  was  decided  that  the 
subjects  would  not  receive  explicit  practice  with,  or  exposure  to, 
the  test  scenarios  in  order  to  maintain  motivation  during  the 
repeated  testing.  This  decision  introduced  the  possibility  that 
workload  ratings  could  be  affected  by  familiarization  with  the 
scenarios  during  testing,  and  that  these  effects  would  confound  the 
impact  of  equipage  levels  and  their  comparison  to  the  baseline 
condition.  As  noted  earlier,  an  attempt  was  made  to  provide  a 
partial  control  for  this  confounding  by  testing  the  two  groups  with 
different  orders  of  exposure  to  the  Data  Link  and  non-Data  Link 
conditions . 

To  address  the  testing  sequence  effect,  the  ANOVA  was  repeated  with 
test  order  substituted  for  the  equipage  factor.  The  SWAT  scores 
are  plotted  to  show  this  effect  in  figure  B-3 .  The  apparent 
decrease  in  perceived  workload  over  test  trials  that  can  be  seen 
in  figure  B-3  was  confirmed  by  the  analysis.  The  test  order  factor 
was  significant  (F ( 2 , 12 ) =5 . 53 ,  p<.05).  No  other  main  effects  or 
interactions  were  statistically  significant. 

Figure  B-4  clearly  shows  the  apparent  dominance  of  the  scenario 
familiarization  effect  on  workload  ratings.  Since  the  70  percent 
Data  Link  condition  was  presented  in  the  second  position  of  the 
test  sequence  for  both  groups  in  both  scenarios,  this  figure 
focuses  on  the  baseline  0  and  2  0  pe*^t=nt  equipage  levels  which 
alternated  as  the  first  and  last  conditions  within  each  scenario. 
As  figure  B-4  indicates,  the  primary  determinant  of  which  of  the 
two  conditions  received  the  greater  workload  rating  was  the  order 
of  presentation:  if  Data  Link  was  used  first  in  a  scenario,  it  was 
rated  as  the  more  difficult  test;  if  baseline  was  tested  first,  its 
workload  was  greater  than  Data  Link. 

This  result  suggests  that  the  original  three-way  interaction 
obtained  in  the  primary  analysis  was  largely  attributable  to  the 
sequence  effect  which  was  reflected  in  the  combination  of  the 
group,  scenario  and  equipage  factors.  More  importantly,  this 
finding  confirms  and  extends  the  results  of  the  first  analysis,  and 
indicates  that: 

1.  Controller  workload  was  unaffected  by  the  substitution  of  Data 
Link  procedures  for  current  voice  radio  procedures  (for  the  ATC 
service  designs  tested) . 

2.  Controller  workload  did  not  differ  in  the  low  (20  percent)  and 
high  (70  percent)  Data  Link-equipage  environments. 
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FIGURE  B-3 .  TEST  SEQUENCE  EFFECT  ON  CONTROLLER  WORKLOAD 
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3.  Occasional  observed  differences  in  workload  between  voice  and 
Data  Link  tests  were  not  statistically  significant  and  were 
attributable  to  scenario  familiarity,  regardless  of  the  procedures 
used. 

A  final  statistical  test  performed  on  the  SWAT  workload  data 
compared  the  70  percent  Data  Link  equipage  test  run  under 
scenario  1  with  a  seventh  test  condition.  This  condition  was 
identical  to  the  former  test,  but  doubled  average  Data  Link  time 
delays  to  a  predicted  "worse  case"  level.  The  mean  SWAT  score  for 
the  combined  groups  in  the  normal  delay  condition  was  17.86,  while 
in  the  extended  delay  run  the  mean  score  was  16.06.  No 
statistically  significant  difference  between  the  means  was  detected 
(t(7)=.278,  p> . 3 ) . 

RATINGS  OF  OPERATIONAL  EFFECTIVENESS 

After  completing  all  seven  test  runs,  each  subject  rated  the  tested 
Data  Link  service  designs  in  terms  of  how  well  they  would  meet 
operational  requirements  under  the  full  range  of  ATC  environments 
and  variations.  The  subjects  were  instructed  to  rate  the 
effectiveness  of  each  design  independently  of  whether  they  felt  it 
was  easy  to  use  or  would  be  acceptable  to  controllers.  The  rating 
scale  permitted  the  subjects  to  classify  a  service  as  "not 
operationally  suitable"  or  on  a  scale  ranging  from  1  (highly 
effective)  to  7  (minimally  effective) . 

None  of  the  services  were  rated  "not  operationally  suitable"  by  any 
of  the  subjects.  The  median  ratings  for  each  service  are  shown  in 
figure  B-5.  Wilcoxon  nonparametric  statistical  tests  indicated 
that  Transfer  of  Communication  (TOC) ,  Altitude  Assignment  (AA)  and 
Menu  Text  for  Interim  Altitudes  (MT)  were  all  rated  higher  than  the 
scale  midpoint  (meets  most  operational  requirements) .  Additional 
tests  showed  that  TOC  and  MT  were  equally  effective  designs  and 
that  both  of  these  were  more  effective  than  AA.  AA  was 
significantly  more  effective  than  Free  Text  (FT) .  Though  rated 
lowest  of  the  group,  FT  was  not  rated  significantly  lower  than  the 
scale  midpoint. 

RATINGS  OF  CONTROLLER  ACCEPTANCE/PREFERENCE 

The  subjects  were  also  asked  to  classify  each  of  the  tested  service 
designs  according  to  how  acceptable  the  procedures,  displays  and 
inputs  would  be  to  controllers.  Subjects  were  instructed  to  make 
this  rating  independently  from  their  evaluation  of  the  perceived 
effectiveness  of  the  design.  The  rating  scale  permitted  the 
subjects  to  classify  a  service  as  "completely  unacceptable"  or  on 
a  preference  scale  ranging  from  1  (highly  preferred)  to  7 
(acceptable,  but  not  preferred) . 

None  of  the  tested  services  were  rated  as  "completely  unacceptable" 
by  any  of  the  subjects.  The  median  preference  ratings  are  shown 
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in  figure  B-5.  Wilcoxon  tests  indicated  that  TOC,  AA,  and  MT  were 
all  rated  higher  than  the  scale  midpoint  (moderately  preferred) . 
Furthermore,  TOC  was  significantly  more  preferred  than  AA,  which 
was,  in  turn,  more  highly  preferred  than  MT.  Though  rated 
significantly  less  preferred  than  the  other  three  services,  FT  was 
not  rated  significantly  lower  than  the  scale  midpoint. 

Controller  comments  were  solicited  on  the  SWAT  rating  forms  and  on 
"incident  forms,"  which  were  completed  following  each  simulation 
test  run.  Additional  comments  were  recorded  by  controllers  while 
completing  the  ratings  of  effectiveness  and  preference  for  each 
service.  Because  similar  comments  were  obtained  on  the  inter¬ 
trial  and  post-test  data  sheets,  they  were  analyzed  as  a  single 
group.  The  following  are  brief,  categorical  summaries  of  these 
comments: 

TESTING  SEQUENCE. 

As  noted  in  the  presentation  of  the  SWAT  data,  the  order  of  testing 
within  a  scenario  appeared  to  profoundly  affect  the  subjects' 
perceived  workload,  regardless  of  whether  a  test  involved  voice 
radio  procedures  or  Data  Link.  Subject  comments  strongly  support 
the  conclusion  that  any  cases  of  increased  workload  under  Data  Link 
or  Baseline  conditions  were  attributable  to  unfamiliarity  with  the 
test  scenario.  A  total  of  nine  comments  which  referred  to  the 
relationship  between  the  test  scenario  and  workload  were  recorded. 
Three  of  the  remarks  indicated  that  the  workload  of  the  scenario 
was  high.  All  of  these  were  recorded  following  the  first  test  run 
within  a  scenario  (regardless  of  equipage) .  Six  comments  indicated 
that  the  preceding  run  had  been  especially  easy  to  deal  with.  All 
six  were  recorded  in  reference  to  the  final  run  with  a  scenario. 

GENERAL  COMMENTS  ABOUT  DATA  LINK. 

A  total  of  17  comments  were  received  regarding  the  general  impact 
of  Data  Link  on  the  ATC  task.  All  of  these  were  positive  comments. 
The  most  common  remarks  referred  to  reduced  frequency  congestion, 
reduced  requirements  for  responding  to  pilot  check-in  and  for 
monitoring  for  voice  responses  to  clearances,  general  workload 
reduction  and  more  available  time,  and  reduced  chance  for  pilot 
interpretation  errors.  Other  comments  noted  the  high  frequency 
with  which  the  subject  was  able  to  employ  Data  Link. 

SERVICE  DESIGNS. 

One  subject  noted  that  the  Data  Link  equipage  symbologies  in  the 
Full  Data  Block  (FDB)  may  be  confusing  to  controllers.  He 
indicated  that  the  diamond  symbol  may  be  misinterpreted  as  a  "flat 
track"  symbol,  and  that  the  hourglass  appeared  similar  to  a 
formerly  used  symbol  denoting  that  another  sector  has  track  control 
and  hold. 
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Two  subjects  indicated  that  an  improved  display  may  enhance  the 
d  ectability  by  a  receiving  controller  of  the  change  from  the 
diamond  to  the  hourglass  symbol  (after  a  TOC) .  As  noted  by  one  of 
the  subjects,  under  the  current  voice  system  the  pilot  check-in 
cues  the  controller  that  the  aircraft  is  ready  to  receive 
instructions.  In  the  Data  Link  system,  the  controller  must  monitor 
closely  for  the  change  to  the  hourglass  symbol.  Thus,  this  change 
in  Data  Link  eligibility  state  may  require  a  more  obvious  and 
salient  signal  to  prevent  excessive  task  demands  on  the  controller. 

One  controller  was  particularly  concerned  with  the  "FAIL"  message 
in  the  FDB.  Since  this  message  does  not  discriminate  among  a  pilot 
"UNABLE"  response,  a  failed  technical  acknowledgement  (NAK)  or  a 
timeout,  he  found  himself  attempting  to  repeat  uplinks  when,  in 
fact,  the  pilot  had  unabled  the  clearance.  He  suggested 
eliminating  the  UNABLE  pilot  Data  Link  response  and  developing  a 
standard  procedure  requiring  pilots  to  make  a  voice  radio  call  if 
incapable  of  complying  with  a  controller's  instruction. 

A  final  recommendation  was  provided  by  a  controller  who  indicated 
that  he  sometimes  forgot  whether  he  had  pressed  the  appropriate  key 
to  uplink  an  altitude  (S) .  He  felt  that  such  confusions  would  be 
reduced  by  increasing  the  duration  of  the  "WILCO"  display 
presentation.  A  second  controller  also  noted  the  need  for  more 
visual  feedback  after  an  altitude  assignment  to  reduce  the 
monitoring  demands  of  the  task.  It  should  be  noted  that  this  issue 
may  be  resolved  by  a  design  modification  which  was  in  progress 
during  the  study.  This  modification  will  present  an  alternating 
display  of  the  original  and  newly  assigned  altitudes  until  a  pilot 
WILCO  is  received. 

In  general,  the  problems  noted  during  the  test  bed  sessions  were 
few  in  number.  However,  they  tended  to  center  around  needed 
reductions  in  the  visual  monitoring  demands  of  completing  Data  Link 
transactions . 

EFFECTS  OF  DATA  LINK  EQUIPAGE  LEVEL. 

The  SWAT  data  indicated  that  there  were  no  significant  differences 
in  workload  between  the  70  and  20  percent  levels  of  aircraft 
equipage.  This  finding  was  supported  by  controller  comments.  Four 
remarks  were  recorded  in  reference  to  this  variable.  All  noted 
that  neither  low  nor  high  densities  of  equipped  aircraft  had  a 
negative  impact  on  the  controller. 

EXTENDED  DATA  LINK  DELAYS. 

The  workload  data  revealed  no  difference  between  normal  and 
extended  Data  Link  delays.  This  result  was  also  supported  by 
controller  remarks.  Four  subjects  commented  on  delay  effects.  All 
indicated  that  they  did  not  feel  the  extended  delay  had  a 
significant  impact  on  their  performance. 
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USE  OF  VOICE  RADIO  WITH  DATA  LINK  EQUIPPED  AIRCRAFT. 

Three  subjects  commented  that  they  had  occasionally  used  radio 
contacts  with  equipped  aircraft.  In  all  cases,  these  subjects 
noted  that  the  reversion  to  voice  was  made  only  during  time- 
critical  situations  when  rapid  compliance  was  required.  None  of 
the  subjects  reported  problems  in  shifting  between  voice  and  data 
link  as  required  by  availability  of  the  link  or  by  time 
constraints . 

FREE  TEXT. 

Although  not  explicitly  instructed  to  do  so,  several  subjects 
indicated  that  they  had  used  the  free  text  service  to  issue  control 
instructions.  Two  subjects  noted  that  they  had  used  free  text 
successfully  for  heading  changes.  However,  as  expected,  one 
subject  indicated  that  free  text  required  excessive  keyboard  inputs 
for  regular  use.  One  subject  also  noted  that  he  mistyped  a  free 
text  control  instruction  which  caused  pilot  confusion.  A  third 
subject  commented  on  the  high  probability  of  controller  and  pilot 
errors  when  free  text  was  used  to  issue  ATC  instructions.  These 
remarks  indicate  that  the  lack  of  error  checking  and  high  input 
demands  make  free  text  primarily  suitable  for  its  originally 
intended  purpose  of  emergency  communications  or  for  use  by  a  "D" 
controller.  The  subject's  spontaneous  use  of  free  text  for  ATC 
instructions,  and  their  subsequent  discovery  of  its  unsuitability 
for  this  purpose,  also  appears  to  explain  the  lower  effectiveness 
and  preference  ratings  received  by  this  service  compared  to  the 
other  three  data  link  functions  tested. 

ATC  INCIDENTS. 

An  Incident  Comments  form  was  available  to  the  subjects  and  the 
facilitator  controllers  after  each  test  run  to  record  any  observed 
events  which  may  have  impacted  the  overall  safety  and  efficiency 
of  ATC.  No  incidents  were  reported  on  any  of  the  trials  which 
would  indicate  a  difference  in  ATC  performance  or  effectiveness 
between  current,  baseline  conditions  and  the  Data  Link  conditions. 

SWAT  WORKLOAD  RATING  INFORMATION 

Attached  to  this  appendix  are  instructions  that  were  issued  to  the 
controller  subjects  prior  to  actual  testing. 


B-13 


REFERENCES 


1.  Reid,  G.  B.  and  Nygren,  T.E.,  "The  Subjective  Workload 
Assessment  Technique:  A  Scaling  Procedure  For  Measuring  Mental 
Workoad,"  in  P.  A.  Hancock  and  N.  Meshkati  (Eds.)  Human  Mental 
Workload .  pp  185-218,  1988. 

2.  Talotta,  et  al.,  Controller  Evaluation  of  Initial  Data  Link  Air 
Traffic  Control  Services:  Mini  Study  1.  FAA  Technical  Center,  1988. 

3.  Talotta,  et  al.,  Controller  Evaluation  of  Initial  Data  Link  Air 
Traffic  Control  Services:  Mini  Study  2.  FAA  Technical  Center,  1989. 


B-14 


ATTACHMENT 


SWAT  WORKLOAD  RATING  INSTRUCTIONS 

A  major  goal  of  this  study  is  to  obtain  expert  opinions  from  air 
traffic  controllers  about  the  usefulness  of  Data  Link  for  providing 
a  selected  group  of  ATC  services.  While  we  will  be  asking  you  for 
a  wide  variety  of  comments  during  the  study,  a  major  focus  will  be 
the  quantitative  data  that  we  collect  about  controller  workload 
from  a  rating  scale  that  you  will  be  asked  to  complete  after  each 
test  run  in  the  simulation  facility.  The  purpose  of  this  document 
is  to  familiarize  you  with  the  scale  that  will  be  used  to  make  your 
workload  ratings,  and  to  describe  a  special  procedure  that  we  will 
ask  you  to  perform  which  will  help  us  to  interpret  your  ratings  on 
the  scale.  The  material  presented  here  will  be  reviewed  in  a 
briefing  that  you  will  receive  before  we  start  the  simulation  runs. 

WORKLOAD  RATINGS. 

After  each  test  run  you  will  be  asked  to  complete  a  simple  rating 
scale  that  measures  the  level  of  workload  that  you  actually 
experienced  while  controlling  traffic  during  the  run.  The  scale 
that  we  will  be  using  for  the  workload  ratings  is  known  as 
Subjective  Workload  Assessment  Technique  SWAT.  SWAT  was  developed 
as  a  method  for  collecting  quantified  data  on  how  hard  a  human 
operator  has  to  work  in  accomplishing  his  job  duties  using 
different  procedures,  equipmert,  etc.,  to  perform  them. 

If  you  examine  the  scale  in  figure  1  you  will  notice  that  SWAT 
defines  workload  in  terms  of  a  combination  of  three  different 
dimensions  that  contribute  to  the  subjective  feeling  of  "working 
hard."  A  workload  rating  in  SWAT  is  accomplished  by  selecting  a 
"1,"  "2,"  or  "3"  on  EACH  of  the  three  scales  representing  the 
dimensions  of  TIME  LOAD,  MENTAL  EFFORT,  and  PSYCHOLOGICAL  STRESS 
LOAD. 

Each  of  these  dimensions  and  their  levels  are  described  below: 
TIME  LOAD. 

Time  Load  refers  to  the  fraction  of  the  total  time  that  you  are 
busy.  When  Time  Load  is  low,  sufficient  time  is  available  to 
complete  all  of  your  mental  work  with  some  time  to  spare.  As  Time 
Load  increases,  spare  time  drops  out  and  some  aspects  of 
performance  overlap  and  interrupt  one  another.  This  overlap  and 
interruption  can  come  from  performing  more  than  one  task  or  from 
different  aspects  of  performing  the  same  task.  At  high  levels  on 
Time  Load,  several  aspects  of  performance  often  occur 
simultaneously,  you  are  busy,  and  interruptions  are  very  frequent. 

Time  Load  may  be  rated  on  the  three-point  scale  below: 
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TIME  LOAD 


1.  Of  tan  have  apart  tima.  Intaruptiona  or  ovarlap 

amono  activltlaa  occur  Infrequently  or  not  at  all. 

2.  Occasionally  have  spare  time.  Interuptiona  or 

ovarlap  among  activltlaa  occur  frequently. 

3.  Almost  never  have  spare  tima.  Intaruptiona  or 

ovarlap  amono  activities  are  vary  frequent  or 
occur  all  the  tima. 

MENTAL  EFFORT  LOAD 

1.  Vary  little  conscious  mental  effort  or 

concentration  required.  Activity  la  almoet 
automatic,  requiring  little  or  ho  attention. 

2.  Moderate  conacioua  mental  effort  or  concentration 

required.  Complexity  of  activity  la  moderately 
high  due  to  uncertainty,  unpredictability,  or 
unfamiliarity.  Considerable  attention  la 
required. 

3.  Extensive  mental  effort  and  concentration 

are  neoeaaary.  Very  complex  activity 
requiring  total  attention. 

PSYCHOLOGICAL  8TRE88  LOAD 

1.  'Little  confusion,  risk,  frustration  or  anxiety 

and  can  easily  be  aocomodated. 

2.  Moderate  stress  due  to  confusion,  frustration  or 

anxiety.  Noticeably  adds  to  workload. 

Significant  compensation  Is  required  to  maintain 
adequate  performance. 

3.  High  to  very  intense  stress  due  to  confusion, 

frustration  or  anxiety.  High  to  extreme 
determination  and  self-control  required. 
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1.  Often  have  spare  time.  Interruptions  or  overlap  among 
activities  occur  infrequently  or  not  at  all. 

2.  Occasionally  have  spare  time.  Interruptions  or  overlap 
among  activities  occur  frequently. 

3.  Almost  never  have  spare  time.  Interruptions  or  overlap 
among  activities  are  very  frequent,  or  occur  all  the  time. 

MENTAL  EFFORT  LOAD. 

As  described  above,  Time  Load  refers  to  the  amount  of  time  one  has 
available  to  perform  a  task  or  tasks.  In  contrast.  Mental  Effort 
Load  is  an  index  of  the  amount  of  attention  or  mental  effort 
required  by  a  task  regardless  of  the  number  of  tasks  to  be 
performed  or  any  time  limitations.  When  Mental  Effort  Load  is  low, 
the  concentration  and  attention  required  by  a  task  is  minimal  and 
performance  is  nearly  automatic.  As  the  demand  for  mental  effort 
increases  due  to  task  complexity  of  the  amount  of  information 
which  must  be  dealt  with  in  order  to  perform  adequately,  the  degree 
of  concentration  and  attention  required  increases.  High  Mental 
Effort  Load  demands  total  attention  or  concentration  due  to  task 
complexity  or  the  amount  of  information  that  must  be  processed. 

Mental  Effort  Load  may  be  rated  using  the  three-point  scale  below: 

1.  Very  little  conscious  mental  effort  or  concentration 
required.  Activity  is  almost  automatic,  requiring  little  or  no 
attention. 

2.  Moderate  conscious  mental  effort  or  concentration  required. 

Complexity  or  activity  is  moderately  high  due  to  uncertainty, 
unpredictability,  or  unfamiliarity.  Considerable  attention 

required. 

3.  Extensive  mental  effort  and  concentration  are  necessary. 
Very  complex  activity  requiring  total  attention. 

PSYCHOLOGICAL  STRESS  LOAD. 

Psychological  Stress  Load  refers  to  the  contribution  to  total 
workload  of  any  conditions  that  produce  anxiety,  frustration,  or 
confusion  while  performing  a  task  or  tasks.  At  low  levels  of 
stress,  one  feels  relatively  relaxed.  As  stress  increases, 
confusion,  anxiety,  or  frustration  increase  and  greater 
concentration  and  determination  are  required  to  maintain  control 
of  the  situation. 

Psychological  Stress  Load  may  be  rated  on  the  three  point  scale 
below: 
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1.  Little  confusion,  risk,  frustration,  or  anxiety  exists  and  can 
be  easily  accommodated. 

2.  Moderate  stress  due  to  confusion,  frustration,  or  anxiety 
noticeably  adds  to  workload.  Significant  compensation  is 
required  to  maintain  adequate  performance. 

3.  High  to  very  intense  stress  due  to  confusion,  frustration, 
or  anxiety.  High  to  extreme  determination  and  self-control 
required. 

Each  of  the  three  dimensions  just  described  contribute  to  workload 
during  performance  of  a  task  or  group  of  tasks.  Note  that  although 
all  three  factors  may  be  correlated,  they  need  not  be.  For 
example,  one  can  have  many  tasks  to  perform  in  the  time  available 
(high  Time  Load)  but  the  tasks  may  require  little  concentration 
(low  Mental  Effort  Load) .  Likewise,  one  can  be  anxious  and 
frustrated  (high  Stress  Load)  and  have  plenty  of  spare  time  between 
relatively  simple  tasks.  Since  the  three  dimensions  contributing 
to  workload  are  not  necessarily  correlated,  please  treat  each 
dimension  individually  and  give  independent  assessments  of  the  Time 
Load,  Mental  Effort  Load,  and  Psychological  Stress  Load  that  you 
feel  were  imposed  during  each  test  run. 

The  form  that  you  will  be  using  to  make  your  SWAT  ratings  during 
the  Data  Link  test  session  is  shown  in  figure  2.  Note  that  the 
descriptions  for  each  level  of  time,  effort,  and  stress  load  have 
been  removed  to  save  space.  Should  you  need  to  review  these 
descriptions  during  testing,  a  copy  of  the  full  scale  will  be 
available  at  all  times. 

SWAT  SCALE  DEVELOPMENT  CARD  SORT. 

Now  that  you  are  familiar  with  rating  scale  that  will  be  used 
during  the  Data  Link  Test  sessions,  there  is  one  last  procedure 
that  must  be  completed  before  testing  can  begin.  This  procedure 
is  a  card  sorting  task  that  will  allow  us  to  interpret  your  SWAT 
workload  ratings.  We  will  be  asking  to  do  this  task  during  the 
briefing  that  will  take  place  to  review  the  rating  scales. 

One  of  the  most  important  features  of  SWAT  is  its  unique  scoring 
system.  The  developers  of  SWAT  recognized  that  different  people 
have  different  conceptions  of  how  the  time,  effort  and  stress 
dimensions  combine  to  produce  an  overall  impression  of  low  and  high 
workload.  Because  of  these  differences,  a  special  card  sorting 
procedure  is  used  in  SWAT  to  define  a  distinctive  workload  scale 
for  each  person.  This  individualized  scale  greatly  improves  our 
ability  to  accurately  interpret  the  actual  workload  ratings  that 
you  will  making  during  the  test  sessions. 

In  order  to  develop  your  individual  scale,  we  need  information  from 
you  regarding  the  amount  of  workload  that  you  feel  is  produced  by 
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NAME  . 

SECTOR  . . 

TEST  CONDITION: 

SCENARIO  1  —  2  ... 

EQUIPAGE  LEVEL  20%  70% 

0%  (VOICE  ONLY)  ... 


Rate  the  workload  that  vou  actually  experienced  while 
controlling  air  traffic  during  this  test  run  by  placing 
an  *X*  below  one  of  the ‘.numbers  on  the  TIME,  EFFORT 
and  STRESS  scales. 

(Low)  (High) 

1  2  3 

TIME  LOAD  . . 

MENTAL  EFFORT  . 

STRESS  . . 


COMMENTS: 
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various  combinations  of  the  three  levels  of  the  time,  effort,  and 
stress  dimensions.  We  get  this  information  by  having  a  person  rank 
order  a  set  of  cards.  Each  card  contains  a  different  combination 
of  the  levels  of  Time  Load,  Mental  Effort  Load,  and  Stress  Load. 
Since  there  are  three  dimensions,  and  each  dimension  has  three 
levels,  there  are  27  cards  in  the  deck  that  you  will  be  sorting. 
Your  job  will  be  to  sort  the  cards  so  that  they  are  ranked 
according  to  the  level  of  workload  represented  by  each  card.  Thus, 
the  first  card  in  the  deck  will  represent  the  lowest  workload  and 
the  last  card  will  represent  the  highest  workload. 

In  completing  your  card  sorts,  please  consider  the  workload  imposed 
on  a  person  by  the  combination  represented  in  each  card.  Arrange 
the  cards  from  the  lowest  workload  condition  through  the  highest 
condition.  You  may  use  any  strategy  you  choose  in  rank  ordering 
the  cards.  One  strategy  that  proves  useful  is  to  arrange  the  cards 
into  a  number  of  preliminary  stacks  representing  "High," 
"Moderate,"  and  "Low"  workload.  Individual  cards  can  be  exchanged 
between  stacks,  if  necessary,  and  then  rank  ordered  within  stacks. 
Stacks  can  then  be  recombined  and  checked  to  be  sure  that  they 
represent  your  ranking  of  lowest  to  highest  workload.  However,  the 
choice  of  strategy  is  up  to  you  and  you  should  choose  the  one  that 
works  best  for  you. 

There  is  no  "school  solution"  to  this  problem.  There  is  no  correct 
order.  The  correct  order  is  what,  in  your  judgement,  best 
describes  the  progression  of  workload  from  lowest  to  highest  for 
a  general  case  rather  than  any  specific  event.  That  judgement 
differs  for  each  of  us.  The  letters  you  see  on  the  back  of  the 
cards  are  to  allow  us  to  arrange  the  cards  in  a  previously 
randomized  sequence  so  that  everyone  -  starts  with  the  same  order. 
If  you  examine  your  deck  you  will  see  the  order  on  the  back  runs 
from  A  through  Z  and  then  ZZ. 

Please  remember: 


1.  The  card  sort  is  being  done  so  that  a  workload  scale  may  be 
developed  for  you.  This  scale  will  have  a  distinct  workload  value 
for  each  possible  combination  of  Time  Load,  Mental  Effort  Load,  and 
Psychological  Stress  Load.  The  following  example  demonstrates  the 
relationship  between  the  card  sort  and  the  resulting  workload 
scale: 


TIME  EFFORT  STRESS  WORKLOAD  SCALE 

11  1  0.0 


3 


3 


3 


100 


B-2  0 


Note  that  other  than  the  fact  that  a  1-1-1  will  always  represent 
the  lowest  workload  and  that  a  3-3-3  will  always  represent  the 
highest  workload,  the  remaining  cards  could  occur  in  a  number  of 
orders.  Your  order  will  depend  on  how  you  weight  the  importance 
of  Time,  Effort,  and  Stress  dimensions. 

2.  When  performing  the  card  sorts,  use  the  descriptors  printed 
on  the  cards.  Please  remember  not  to  sort  the  cards  based  on  one 
particular  task  (such  as  controlling  air  traffic) .  Sort  the  cards 
according  to  your  general  view  of  workload  and  how  important  you 
consider  the  dimensions  of  Time,  Mental  Effort,  and  Psychological 
Stress  Load  to  be. 

3.  During  the  Data  Link  test  runs,  you  will  accomplish  the 
desired  task.  Then,  you  will  provide  a  SWAT  rating  based  on  your 
opinion  of  the  mental  workload  that  you  experienced  while 
performing  the  task.  This  SWAT  rating  will  consist  of  one  number 
from  each  of  the  three  dimensions.  For  example,  a  possible  SWAT 
rating  is  1-2-2.  This  represents  a  1  for  Time  Load,  a  2  for  Mental 
Effort  Load,  and  a  2  for  Psychological  Stress  Load. 

4.  When  doing  the  sorting  task,  we  are  not  asking  for  your 
preference  concerning  Time,  Mental  Effort,  and  Psychological  Stress 
Load.  Some  people  may  prefer  to  be  '•busy"  rather  than  "idle"  in 
either  the  Time  Load,  Mental  Effort  Load,  or  Psychological  Stress 
Load  dimension.  We  are  not  concerned  with  this  preference.  We 
need  information  on  how  the  three  dimensions  and  the  three  levels 
of  each  one  will  affect  the  level  of  workload  as  you  see  it.  You 
may  prefer  a  2-2-2  situation  instead  of  a  1-1-1  situation. 
However,  you  should  still  realize  that  the  1-1-1  situation  imposes 
less  workload  on  you  and  leaves  a  greater  reserve  capacity. 

The  sorting  will  probably  take  30  minutes  to  an  hour.  Please  feel 
free  to  ask  questions  at  any  time. 
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INTRODUCTION 


During  the  operational  evaluation  of  the  initial  Air  Traffic 
Control  (ATC)  Data  Link  services,  various  types  of  data  were 
recorded  by  the  computer  systems  used  in  the  tests.  The  data  were 
collected  and  reduced  using  data  reduction  tools  available  on  the 
Host  Computer  System  and  other  data  reduction  methods  developed  by 
MITRE  personnel.  The  reduced  data  were  then  input  to  a  graphics 
package  to  produce  a  set  of  graphical  results  depicting  Data  Link 
metrics. 

The  Data  Link  data  analysis  results  provided  herein  consist  of  a 
number  of  metrics  presented  on  graphs  that  provide  information 
about  specific  aspects  of  the  controller  Data  Link  and  voice 
communication  message  activity  which  took  place  during  the  test 
runs.  The  data  analysis  results  are  divided  into  three  sections. 
The  first  section,  "Data  Link  Activity  Results,"  reports  on  the 
Data  Link  activity  which  took  place  during  the  testing.  The  second 
part,  "Voice  Communications  Results,"  describes  the  effect  Data 
Link  had  on  voice  communications,  while  the  third  section,  "Data 
Link  and  Voice  Message  Results,"  compares  the  message  activity  of 
voice  and  Data  Link. 

The  sources  of  the  data  used  to  produce  the  metrics  were  the  Host 
Computer  System,  which  logged  all  Data  Link  activity,  and  the 
AMECOM  FAA  Technical  Center  Communication  Switching  System,  which 
recorded  all  controller  voice  communications.  Data  pertaining  to 
pilot  voice  communications  were  not  available;  therefore,  all  data 
reported  for  voice  communications  applies  only  to  controller 
operations. 

Two  different  scenarios  were  used  in  the  evaluations,  each  having 
a  0,  20,  and  70  percent  Data  Link-equipage  level.  The  sequence  of 
the  runs,  the  scenario  used,  and  the  Data  Link-equipage  are 
provided  in  figure  C-l.  (Please  note;  Only  test  runs  that  were  used 
for  data  reduction  are  included  in  this  chart.  Other  test  runs 
conducted  in  the  operational  evaluation  were  intended  for  special 
study  of  specific  Data  Link  aspects  and  were  not  appropriate  for 
this  analysis) . 

The  order  of  the  runs  and  the  Data  Link-equipage  are  presented 
because  of  the  influence  they  had  on  the  data  collected.  An 
ordering  effect  (i.e.,  which  order  the  Data  Link-equipage  scenario 
was  presented)  became  significant  as  a  result  of  a  learning  curve 
experienced  by  the  controllers.  During  the  first  run  of  a  scenario 
the  controllers  were  unfamiliar  with  the  traffic  pattern,  but,  by 
the  third  run  with  the  same  scenario,  the  controllers  knew  what  to 
expect,  and,  therefore,  did  not  need  to  communicate  with  the 
aircraft  as  frequently.  This  effect  is  apparent  in  some  of  the  data 
and  should  be  noted  and  factored  into  the  interpretation  of  the 
results.  Additionally,  due  to  the  varied  lengths  of  the  test  runs, 
all  data  collected  are  based  on  the  same  half-hour  interval  (System 
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FIGURE  C-l.  SEQUENCE  OF  DATA  LINK  TEST  RUNS 


Time  -  12:55  to  13:25).  This  time  interval  was  chosen  because  the 
shortest  test  run  ended  at  approximately  13:25.  Therefore,  the  data 
collected  represents  the  same  time  intervals  in  the  test  runs. 

DATA  LINK  ACTIVITY  RESULTS 

This  group  of  results  measures  the  Data  Link  activity.  These 
results  contain  four  figures  which  represent  the  average  Data  Link 
activity  and  traffic  volume  which  took  place  during  the  test  runs 
listed  in  figure  C-l.  The  average  consists  of  the  message  activity 
and  traffic  volume  during  the  eight  Data  Link  runs  at  2  0  and  7  0 
percent  Data  Link-equipage  divided  by  8.  For  example,  all  uplinks 
for  the  eight  Data  Link  runs  are  added  together  and  divided  by  8 
to  give  the  102  uplinks  in  the  bar  chart  in  figure  C-2 . 

Figure  C-2  presents  the  Uplinks,  Will  Complies  (WILCOs) ,  No 
Technical  Acknowledgements  (NAKs) ,  and  No  Pilot  Responses  or  Pilot 
Unables  which  are  both  denoted  by  FAIL  (i.e.,  a  FAILed  response 
from  the  pilot)  .  The  bar  chart  on  the  left  contains  the  average 
number  of  messages  in  each  of  the  above  categories,  while  the  pie 
chart  on  the  right  shows  the  WILCOs,  NAKs,  and  FAILS  as  a 
percentage  of  uplinked  messages. 

On  the  average,  102  Lata  Link  messages  were  uplinked  during  the 
half-hour  interval  of  each  test.  Also,  from  the  pie  chart,  91 
percent  of  all  Data  Link  messages  were  WILCOed  by  pilots,  and  only 
4  percent  of  the  messages  were  either  "UNABLED"  by  the  pilot,  or 
not  responded  to  within  the  ^40-second  pilot  response  timeout 
period*  (e.g.,  FAIL  messages)***  This  figure  appears  impressive 
with  respect  to  the  number  of  times  a  controller  must  reissue 
instructions  by  voice  because  of  garbled  communications  over  the 
radio  channels.  The  results  indicate  that  Data  Link  messages  are 
sent  to  pilots  with  a  low  repeat  rate  and  that  Data  Link  message 
response  time  occurs  within  an  acceptable  time  period. 

With  respect  to  NAKed  messages,  the  VAX  computer,  acting  as  the 
Data  Link  processor,  generated  the  NAK's  randomly  during  the  tests. 
This  was  done  to  simulate  equipment  failures  which  might  take  place 
during  actual  field  implementation.  The  VAX  was  programmed  to 
generate  NAK's  at  a  5  percent  rate  (this  rate  is  well  above  the 
rate  expected  in  actual  implementation,  but  was  set  to  5  percent 
during  the  tests  so  an  analysis  could  be  performed  on  controller 
reaction  to  this  situation) ;  the  5  percent  NAK  rate  was  verified 
in  the  analysis. 


♦The  40-second  pilot  response  timeout  period  includes  uplink  time, 
pilot  recognition  and  response  time,  and  downlink  time. 

♦♦During  the  operational  evaluation,  simulation  pilots  were  used 
for  the  majority  of  aircraft  flying  in  the  scenarios.  (Only  two 
aircraft  simulators  rate  needs  to  be  studied  with  a  greater 
percentage  of  pilots  flying  real  aircraft.) 
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Figure  C-3  presents  the  average  Data  Link  activity  which  took  place 
for  each  Data  Link  service  or  function  (these  numbers  represent  the 
number  of  uplinked  messages  for  each  service  or  function  regardless 
of  the  downlink  response)  .  The  Data  Link  services  and  functions 
tested  in  the  operational  evaluation  were:  Altitude  Assignment 
(AA)  ,  Interim  Altitude  via  Menu  Text  (MT)  ,  Transfer  of 
Communication  (TC)  ,  and  Free  Text  (FT)  .  The  menu  text  figures 
represent  interim  altitudes  via  the  menu  text  function.  Since  menu 
text  was  used  for  the  sole  purpose  of  uplinking  interim  altitudes 
in  the  operational  evaluation,  no  other  menu  text  figures  are 
given. 

The  bar  chart  provides  the  average  number  of  uplinks  per  service 
or  function  over  the  half-hour  interval.  In  addition,  the 
percentage  use  of  each  of  the  capabilities  is  given  in  the  pie 
chart.  For  example,  on  the  average,  the  Transfer  of  Communication 
service  represented  29  percent  of  all  uplink  messages.  Also,  from 
this  chart  the  combination  of  Altitude  Assignment  and  Interim 
Altitude  via  Menu  Text  represented  57  percent  of  all  uplinks. 

The  third  chart  in  this  group  of  results  (figure  C-4)  displays  the 
Data  Link  activity  by  Uplinks,  WILCOs,  NAKs,  and  FAILs  during  3- 
minute  intervals  during  the  tests.  This  provides  a  look  at  the  Data 
Link  activity  during  specific  times  during  the  test  runs. 

The  traffic  volume  in  the  operational  evaluation  scenarios  varied 
from  sector  to  sector,  but  gradually  increased  for  all  sectors 
between  the  time  interval  12:55  to  13:25  from  which  the  data  were 
taken  (see  figure  C-5) .  The  Uplinks  and  their  responses  presented 
in  figure  C-4  increase  at  approximately  the  same  rate  as  the 
traffic  volume  presented  in  figure  C-5.  Therefore,  the  data  show 
that  Data  Link  can  be  used  with  the  same  effectiveness  during  low 
and  high  traffic  loads. 

VOICE  COMMUNICATIONS  RESULTS 

The  voice  communication  results  are  based  on  digital  recordings  of 
the  controller  voice  activity.  The  computer  systems  used  in  the 
evaluation  did  not  have  the  capability  to  record  pilot  voice 
communications,  therefore,  these  results  are  based  solely  on 
controller  voice  communications.  Even  though  the  pilot  voice 
activity  is  not  included  in  the  results,  the  controller  voice 
communications  can  provide  a  good  indication  of  the  reduction  in 
activity  on  the  radio  channels  as  a  result  of  the  initial  Data  Link 
services. 

The  voice  communication  data  provides  an  analysis  on  the  number 
and  duration  of  controller  voice  messages.  Figures  C-6  and  C-7 
report  on  the  reduction  in  the  number  of  messages,  while  figures 
C-8  and  C-9  provide  data  on  the  reduction  in  duration  of  voice 
messages  during  the  half-hour  interval. 
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FIGURE  C-3.  AVERAGE  DATA  LINK  ACTIVITY  BY  SERVICE/FUNCTION 
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FIGURE  C-5 .  AVERAGE  NUMBER  OF  DATA  LINK.  AIRCRAFT,  3-MINUTE  INTERVALS 
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Figure  C-6  presents  the  average  number  of  voice  messages  during 
the  0,  20,  and  70  percent  Data  Link-equipage  scenarios.  The  average 
represents  the  sum  of  the  number  of  controller  voice  messages 
during  each  run  at  each  Data  Link-equipage  level  divided  by  4 
(i.e.,  there  were  four  0  percents,  four  20  percents,  and  four  70 
percents  runs  in  the  data  collection,  see  figure  C-l) .  For  example, 
during  all  of  the  0  percent  Data  Link-equipage  scenarios  tested, 
the  controller  issued  an  average  of  422  voice  messages  during  the 
half-hour  interval. 

Figure  C-7  compares  the  number  of  voice  messages  during  the 
different  equipage  runs.  The  numbers  given  show  the  reduction  in 
controller  voice  messages  using  two  levels  of  Data  Link-equipage. 
From  the  data  41  percent  of  voice  messages  (i.e.,  the  number  of 
messages)  were  eliminated  when  70  percent  of  the  aircraft  were  Data 
Link-equipped.  (The  41  percent  figure  is  based  on  the  number  of 
voice  messages  during  the  70  percent  Data  Link-equipage  scenario 
as  a  percentage  reduction  in  the  number  of  voice  messages  in  the 
0  percent  Data  Link-equipage  scenario.) 

Figure  C-8  provides  the  average  duration  of"  the  voice  messages 
during  each  of  the  0,  20,  and  7  0  percent  Data  Link-equipage 
scenarios.  These  numbers  represent  the  time,  in  seconds,  the 
controllers  were  on  the  radio  frequencies  during  the  half-hour 
interval.  The  average  was  computed  the  same  as  in  figure  C-5, 
except  the  duration  of  the  messages  was  averaged  instead  of  the 
number  of  messages.  For  example,  on  the  average,  during  the  20 
percent  Data  Link-equipage  scenarios  tested,  the  controllers  were 
on  the  radio  frequencies  for  a  total  of  1046  seconds. 

Figure  C-9  compares  the  voice  messages  during  the  different 
equipage  runs.  The  graph  shows  the  reduction  in  the  total  duration 
of  voice  messages  as  a  result  of  the  two  Data  Link-equipages.  From 
the  data,  45  percent  of  the  voice  communications  (e.g.,  the 
duration  of  the  communications)  were  eliminated  when  70  percent  of 
the  aircraft  were  Data  Link-equipped.  (The  45  percent  figure  is 
based  on  the  duration  of  voice  messages  during  the  70  percent  Data 
Link-equipage  scenario  as  a  percentage  reduction  in  the  duration 
of  voice  messages  in  the  0  percent  Data  Link-equipage  scenario.) 

The  data  given  for  the  20  percent  Data  Link-equipage  seems 
inconsistent  with  the  effect  Data  Link  should  have  on  the  voice 
communications.  From  the  data  in  figure  C-9  it  appears  that  a  20 
percent  Data  Link-equipage  reduces  voice  communications  by  28 
percent.  These  are  the  numbers  reported  by  the  data,  but  the  20 
percent  runs  were  heavily  influenced  by  the  ordering  effect.  The 
ordering  effect  is  the  reduction  in  communications  needed  with  the 
aircraft  as  the  controller  becomes  familiar  with  the  scenario. 
Since  the  controller  knows  where  and  when  the  aircraft  are  present 
in  the  scenario,  by  the  third  run  he  no  longer  needs  to  issue  ATC 
instructions  as  often. 
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From  figure  C-l  the  20  percent  runs  were  presented  to  the 
controllers  either  first  or  third,  while  the  70  percent  runs  were 
always  run  second.  Since  the  70  percent  runs  were  always  run 
second,  the  number  of  times  the  0  percent  runs  were  either  first 
or  third  were  equal,  thus,  eliminating  the  ordering  effect  on  the 
70  percent  runs  when  taking  the  average.  But  the  20  percent  runs 
did  not  experience  this.  Therefore,  it  is  recommended  that  in 
future  testing,  the  ordering  effect  be  reduced  by  improving 
controller  familiarization  with  the  scenario  or  by  making  more  test 
runs  to  balance  out  the  effect. 

In  addition,  to  fully  understand  the  impact  of  Data  Link  on  voice 
communications,  future  tests  need  to  be  performed  with  other  Data 
Link  equipages.  But,  from  the  data  collected  thus  far,  it  is 
apparent  that  Data  Link  will  have  a  significant  impact  on  reducing 
the  number  and  duration  of  voice  communications  on  the  radio 
channels. 


DATA  LINK  AND  VOICE  MESSAGE  RESULTS 

Figure  C-10  presents  the  average  number  of  voice  and  Data  Link 
messages  during  each  of  the  0,  20,  and  70  percent  Data 
Link-equipage  runs.  The  data  are  presented  together  to  show  how  an 
increase  in  Data  Link-equipped  aircraft  produces  a  decrease  in  the 
number  of  voice  messages. 

In  addition,  as  figure  C-10  indicates,  when  Data  Link  is  used,  the 
total  number  of  messages  (voice  plus  Data  Link)  is  reduced.  For 
example,  when  Data  Link  was  not  used,  there  were  422  total 
messages.  But,  when  70  percent  the  aircraft  were  Data 
Link-equipped,  there  were  only  375  total  messages.  An  explanation 
of  these  results  could  be  the  reduction  in  repeat  communications 
via  voice  when  Data  Link  is  used.  Data  Link  messages  need  not  be 
repeated  due  to  misunderstandings  as  is  the  case  with  voice,  where 
the  message  may  become  distorted  on  the  radio  frequency. 

The  data  indicate  that  not  only  does  Data  Link  reduce  voice 
congestion  on  the  radio  channels,  but  also  reduces  the  need  for 
repeat  voice  communications.  Additionally,  Data  Link  affords  a 
safer  ATC  system  by  providing  clearly  understood  ATC  messages. 
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ANALYSIS  OF  GROUP  DISCUSSION  AND  DEBRIEFING  DATA 
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INTRODUCTION 


This  report  analyzes  the  commentary  offered  in  a  structured 
debriefing  discussion  with  the  Air  Traffic  Control  Specialists 
(ATCS's)  who  participated  in  the  operational  evaluation  of  Data 
Link  services.  In  the  context  of  the  study  protocol,  this 
discussion  followed  an  unstructured  joint  debriefing  session  that 
allowed  some  of  the  participating  flight  crews  to  interact  with 
the  ATCS's  by  teleconference. 

The  issues  addressed  in  the  structured  debriefing  were  presented 
in  a  discussion  guide  form  and  handed  out  prior  to  the  session. 
This  analysis  covers  oral  comments  obtained  during  the  session  and 
written  comments  submitted  on  the  discussion  guide  forms. 
Beginning  with  a  background  section  describing  the  participants, 
the  report  then  turns  to  the  three  issue  areas  addressed  in  the 
discussion  guide:  operational  procedures,  displays  and  inputs,  and 
controls. 


PARTICIPANTS 

The  eight  ATCS's  who  operated  control  positions  in  the  operational 
evaluation  reported  background  information  on  their  air  traffic 
control  (ATC)  experience  and  expertise.  In  all,  six  en  route 
facilities  were  represented:  Memphis  Center,  Washington  Center, 
Fort  Worth  Center,  Anchorage  Center,  Miami  Center,  and  Cleveland 
Center.  All  but  two  of  the  ATCS's  were  current  full  performance 
level  (FPL)  controllers.  FPL  experience  ranged  between  2.5  and  33 
years  with  a  mean  of  11  21  years.  Three  of  the  seven  also 
reported  pilot  experience  (mean  of  6.7  years).  Other  related 
expertise  reported  by  the  ATCS's  included  an  airspace  and 
procedures  specialist,  an  Air  Force  ATCS,  and  an  FAA  academy 
instructor. 

Apart  from  the  ATCS's  who  operated  the  control  positions  in  the 
operational  evaluation,  the  en  route  members  of  the  FAA  Air  Traffic 
Data  Link  Validation  Team  (ATDLVT)  also  attended  the  discussion 
session.  The  ATDLVT  members  participated  primarily  as  facilitators 
throughout  the  discussion  session  to  clarify  issues  and  correct 
misunderstandings . 


OPERATIONAL  PROCEDURES 

Operational  procedures  was  the  first  area  taken  up  in  the 
debriefing.  This  area  includes  both  controller-pilot  procedures  and 
controller-computer  procedures.  Four  themes  emerged  from  the 
analysis  of  the  comments  on  procedures:  control  of  message 
transmission  and  content,  exception  handling  and  display  feedback, 
allocation  of  Data  Link  tasks  within  the  radar  team,  and 
refinements  and  enhancements  to  Data  Link  capabilities. 
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CONTROL  OF  MESSAGE  TRANSMISSION  AND  CONTENT. 


According  to  the  ATCS's,  operational  procedures  for  the  initial 
Data  Link,  services  should  contain  specific  guidelines  for  control 
of  Data  Link  message  transmissions.  Under  normal  circumstances, 
the  controller  should  wait  for  operational  acknowledgement  of  one 
Data  Linked  instruction  before  transmitting  a  second  instruction 
to  the  same  aircraft.  This  form  of  control  was  deemed  especially 
important  for  the  transfer  of  communication  (TOC)  message  because 
of  the  associated  transfer  of  Data  Link  communication  eligibility. 
There  was  good  agreement  among  the  ATCS's  that  while  the  TOC  was 
underway  no  other  transmissions  should  be  initiated  to  the 
aircraft.  For  the  most  part,  this  procedure  applies  to  all  other 
ATC  instructions;  however,  the  ATCS's  recognized  that  there  may  be 
special  situations  (e.g.,  onboard  emergency)  where  it  becomes 
necessary  to  transmit  additional  instructions  without  waiting  for 
an  operational  acknowledgement. 

On  the  topic  of  transmission  control,  there  was  some  discussion  of 
implementing  an  automated  control  over  message  transmission  at 
least  for  the  TOC  message.  As  discussed  by  the  ATCS's,  this 
automation  would  effectively  lock-out  all  other  transmissions  to 
an  aircraft  while  a  TOC  was  pending  (i.e.,  sent  but  not  yet 
acknowledged) .  However,  at  this  stage  in  the  development  of  Data 
Link  it  would  seem  prudent  to  leave  such  a  decision  to  the 
controller.  An  automated  procedure  could  have  far  reaching 
implications  for  the  service  designs.  A  number  of  related  issues, 
such  as  impact  on  the  Data  Link  eligibility  override  capability, 
should  be  investigated  before  an  automated  control  over  message 
transmission  is  recommended.  For  now,  a  more  appropriate 
compromise  would  be  an  automated  notification  to  the  controller 
attempting  transmission  that  the  aircraft  has  a  pending 
transaction;  and  if  the  pending  transaction  were  a  TOC,  the 
automation  could  also  require  explicit  confirmation  (using  a 
distinctive  confirm  key)  by  the  controller  before  transmitting  the 
new  instruction. 

There  was  also  some  discussion  of  operational  procedures  to  control 
message  content.  The  ATCS's  agreed  that  explicit  guidelines  should 
be  adopted  to  restrict  the  use  of  free  text  messages.  By 
definition,  free  text  messages  should  be  reserved  for  communication 
needs  beyond  the  capabilities  provided  by  the  available  ATC 
services.  Use  of  free  text  is  labor  intensive  and  can  be  error 
prone.  As  a  rule,  the  ATCS's  recommended  that  this  method  of 
message  composition  not  be  used  for  control  instructions. 

EXCEPTION  HANDLING  AND  DISPLAY  FEEDBACK. 

A  second  theme  that  emerged  in  the  discussion  of  operational 
procedures  concerned  handling  of  exceptions  to  normal  Data  Link 
transactions  and  the  presentation  of  feedback  to  controllers  on  the 
status  of  exceptions.  An  exception  is  defined  as  any  transaction 
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that  is  not  completed  with  a  WILCO  acknowledgement  by  the  flight 
crew.  During  the  operational  evaluation,  controller  procedures  and 
display  feedback  were  premised  on  an  assumption  that  flight  crews 
would  contact  the  controller  on  the  voice  channel  if  they  were 
unable  to  comply  with  an  uplinked  control  instruction.  As  it 
turned  out,  the  participating  flight  crews  often  downlinked  their 
UNABLE  responses  without  an  accompanying  voice  call. 

The  discussions  on  exception  handling  identified  two 
recommendations  to  support  exception  handling.  First,  controller 
and  pilot  procedures  need  to  be  coordinated.  So  far,  use  of  voice 
to  work  out  an  alternative  if  the  pilot  is  unable  to  comply  with 
an  instruction  appears  to  be  most  acceptable  as  a  standard 
procedure.  Second,  controller  display  feedback  needs  to 
differentiate  among  different  exception  conditions:  pilot  UNABLE 
response,  no  pilot  response  within  message  timeout  parameter,  and 
message  delivery  failure.  An  UNABLE  response  option  for  flight 
crews  will  be  required  for  Data  Link  communication,  if  only  to 
update  the  automated  systems  in  the  air  and  on  the  ground  that  the 
last  instruction  is  not  the  current  agreement.  The  downlinked 
UNABLE  could,  in  fact,  be  processed  somewhat  like  the  WILCO 
response  by  the  ground  automation;  clearing  the  controller's  Data 
Link  status  displays,  but  also  restoring  the  data  block  and/or  data 
base  to  reflect  the  previous  values.  Further  discussion  and 
analysis  of  the  UNABLE  response  is  needed  before  suitable  automated 
processing  can  be  specified. 

ALLOCATION  OF  DATA  LINK  TASKS  WITHIN  THE  RADAR  TEAM. 

A  third  theme  from  the  discussion  of  Data  Link  procedures  was  ndar 
associate  controller  tasking.  Recently,  the  concept  of  a  radar 
team  for  operational  position  staffing  was  formally  introduced  with 
the  development  of  the  Operational  Position  Standards  (FAA  7220.2) . 
This  concept  provides  flexibility  in  the  assignment  of  tasks  to  the 
team  of  controllers  who  staff  an  operational  position  (radar 
sector) .  During  the  operational  evaluation  and  in  the  previous 
mini  studies,  a  division  of  Data  Link  tasking  between  the  radar 
position  and  the  radar  associate  position  was  mentioned  repeatedly. 
In  particular,  the  ATCS ' s  felt  that  responsibility  for  TOC's  would 
blend  smoothly  with  the  radar  associate's  current  tasking.  In 
addition,  computer  entries  required  for  composition  of  lengthy  Data 
Link  messages  (route  amendments)  could  also  be  performed  by  the 
radar  associate.  Since  all  of  the  test  results  converge  on  this 
recommendation,  procedures  and  software  accommodating  the  above 
division  of  tasking  should  be  presented  to  the  ATDLVT  for  inclusion 
in  future  tests. 

REFINEMENTS  AND  ENHANCEMENTS  TO  DATA  LINK  CAPABILITIES. 

Under  the  fourth  theme,  Data  Link  refinements  and  enhancements,  a 
number  of  comments  on  functions  and  procedures  are  grouped  for 
discussion.  Several  comments  concerned  the  TOC  service: 
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1.  Redesign  of  the  TOC  message  SENT  status  display  for  the 
receiving  controller  was  recommended.  Instead  of  the  existing  Data 
Link  status  list  entry  on  the  receiving  controller  display,  it  was 
recommended  that  only  the  data  block  display  of  TOC  message  SENT 
symbology  should  be  presented  to  the  receiving  controller. 

2.  Implementation  of  an  automatic  TOC  option  was  also  recommended. 
This  option  has  been  recommended  in  both  of  the  preceding 
mini  studies;  it  has  also  been  discussed  at  some  length  with  the 
ATDLVT.  There  is  good  support  for  the  idea  and  its  design  should 
be  taken  up  with  the  ATDLVT  for  their  concurrence  and  action. 

3.  A  controller  option  to  specify  (in  real  time)  the  default 
frequency  for  the  TOC  message  was  recommended  as  was  a  frequency 
assignment  service.  The  frequency  assignment  service  would  support 
outages  and  multiple  frequency  sectors  by  enabling  the  controller 
to  transmit  a  message--"change  to  my  frequency  _ ." 

4.  After  considerable  discussion,  a  majority  opinion  was  reached 
on  the  issue  of  voice  check-in:  a  voice  check-in  on  frequency  is 
deemed  acceptable  as  a  transitional  procedure  when  Data  Link  is 
first  introduced.  Notwithstanding  the  majority  opinion,  there 
remains  a  strong  minority  opinion  among  the  ATCS ' s  that  this 
transmission  should  be  phased  out  as  soon  as  flight  crew  confidence 
or  frequency  confirmation  via  advanced  avionics  permits. 

A  number  of  comments  concerned  refinement  of  the  existing  interim 
altitude  menu  and  additional  capabilities  for  menu  message 
selection  and  transmission.  The  existing  design  for  the  interim 
altitude  menu  has  two  special  use  menu  items:  "R"  and  "Z."  The 
special  use  menu  item  "R"  is  called  requested  altitude.  Selection 
of  menu  item  "R"  automatically  generates  an  altitude  assignment 
message  that  contains  final  assigned  or  requested  altitude 
currently  stored  in  the  NAS  data  base  for  the  aircraft.  It  was 
recommended  that  this  item  appear  in  a  hard  altitude  assignment 
(QZ)  menu  instead  of  the  interim  altitude  assignment  (QQ)  menu. 

The  "Z"  item,  called  controller-defined  altitude,  was  intended  to 
allow  a  controller  with  Data  Link  eligibility  to  transmit  an 
interim  altitude  assignment  to  an  aircraft  after  track  control  had 
been  handed  off  to  the  next  sector.  It  was  to  be  used  in 
situations  where  the  controller  in  the  next  overlying/underlying 
sector  uses  the  data  block  to  coordinate  with  the  current 
controller  by  entering  the  available  interim  altitude  for  the 
aircraft.  Instead  of  the  existing  procedure  where  the  current 
controller  selects  "z"  and  composes  the  altitude  assignment  message 
based  on  the  interim  altitude  displayed  in  the  data  block,  it  was 
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recommended  that  the  "Z"  item  be  redesigned  to  automatically 
compose  the  message.  Both  "R"  and  "Z"  would  then  function 
identical1 y  except  that  the  "R"  item  would  generate  a  hard  altitude 
assignment  and  the  "Z"  item  would  generate  an  interim  altitude 
assignment.  This  modification  would  reduce  controller  inputs  and 
the  risk  of  input  errors.  It  would  also  ensure  that  the  altitude 
information  displayed  to  the  controller  matches  the  information 
transmitted  to  the  pilot. 

Taken  together,  the  above  recommendations  open  up  the  possibility 
of  further  reducing  controller  inputs.  Transmission  of  the  current 
values  for  assigned  and  interim  altitude  could  be  implied  actions. 
Selection  of  the  QZ  or  QQ  quick  action  key  followed  by  an  "S"  to 
indicate  an  uplink  request  and  the  flight  identification  would 
completely  specify  the  message  content,  obviating  the  need  to  enter 
a  menu  item  label. 

The  final  comment  on  menu  messages  concerned  the  addition  of 
compound  altitude  and  heading  messages  to  the  capability.  Such 
messages  are  commonly  required  to  establish  departing  aircraft  on 
their  routes.  They  are  repetitious  in  nature  and  could  offload 
many  routine  transmissions  from  the  voice  channel. 

DISPLAYS 

As  in  the  previous  studies,  the  ATCS's  who  participated  in  the 
operational  evaluation  reiterated  the  principle  that  to  the  extent 
possible  Data  Linked  control  instructions  and  their  status  should 
be  presented  in  the  full  data  block.  In  addition  to  the  comments 
on  exception  status  displays  that  were  reviewed  in  the  preceding 
section,  the  ATCS's  also  commented  that  the  WILCO  status  display 
was  removed  too  quickly.  A  display  persistence  parameter  of 
6  seconds  was  used  in  the  operational  evaluation,  based  on  what 
were,  essentially,  inconclusive  test  results  from  the  second  mini 
study.  Given  the  comments  from  this  second  group  of  ATCS's,  it 
would  appear  that  a  longer  (12  seconds)  display  persistence  may  be 
a  more  suitable  default  parameter. 

Another  important  display  issue  that  came  up  in  the  operational 
evaluation  was  the  principle  of  display  by  exception.  Again,  this 
issue  has  been  discussed  in  the  earlier  studies,  but  a  consensus 
position  was  not  achieved.  The  intent  of  the  display  by  exception 
principle  is  to  help  manage  the  Data  Link  status  list;  it  does  not 
necessarily  affect  the  data  block  displays.  According  to  the 
ATCS's,  only  "FAILED"  altitude  transactions  should  have  both  a  data 
block  and  a  list  presentation.  A  data  block  presentation  of 
feedback  is  sufficient  for  normal  (WILCOed)  altitude  transactions. 
By  confining  the  list  entries  for  altitude  transactions  to  failed 
transactions,  the  list  could  be  kept  shorter  and,  thereby,  simplify 
controller  monitoring.  Other  suggestions  for  list  management,  such 
as  creation  of  sublists  for  different  services,  have  been  proposed 
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in  previous  discussions.  This  list  management  issue  should  be 
referred  to  the  ATDLVT . 

On  the  topic  of  transmission  control  and  e1  ror  response  feedback, 
the  ATCS's  recommended  a  specific  display  notification.  Two  other 
transmission  error  notifications  were  identified  in  the  previous 
studies:  "not  Data  Link  equipped"  and  "not  your  Data  Link 
control."  Both  of  these  notifications  effectively  prevent 
transmission  of  the  intended  message  and  require  the  controller  to 
reconstruct  an  appropriate  message.  In  the  former  case,  the 
reconstructed  message  is  addressed  only  to  the  NAS  computer;  in  the 
latter  case,  the  controller  may  use  an  eligibility  override 
capability  to  enable  subsequent  transmissions.  A  third  type  of 
notification  was  proposed  in  this  study:  "Data  Link  control 
transfer  in  progress."  This  should  be  presented  to  the  controller 
who  attempts  to  transmit  a  message  to  an  aircraft  while  a  TOC  is 
pending.  In  addition,  an  explicit  controller  confirmation  may  be 
desirable  to  implement  a  transmission  while  the  TOC  is  pending. 

INPUTS  AND  CONTROLS 

On  the  topic  of  Data  Link  controls,  the  ATCS's  gave  two 
recommendations.  First,  the  display  of  the  menu  message  list 
should  be  under  a  separate  control  from  display  of  the  message 
status  list.  Second,  a  capability  to  transmit  the  TOC  message  by 
trackball  selection  should  be  provided.  Both  recommendations  have 
been  cited  in  previous  studies  and  design  proposals  should  be 
presented  to  the  ATDLVT. 

Finally,  the  ATCS's  recommended  that  the  initial  Data  Link 
capability  should  be  broadened  to  include  a  full  range  of  control 
instructions.  Aside  from  altitude  and  frequency  assignments, 
heading  and  route  and  speed  and  delay  instructions  were  recommended 
as  initial  Data  Link  services. 


D-6 


ATTACHMENT 


1  OPERATIONAL  EVALUATION  OF  DATA  LINK 

SERVICES 

DEBRIEFING  DISCUSSION  GUIDE 

*ic1t1t1t*it1ticicic1cicicjcicicit*icicjc'k4c'kiticit'k'k’k’k'kit’k1t'kit'k1t'kit‘k'k'k’k'k'k‘kicit-k‘k'k-k-k'k'kicjc'k’kicic,k 

2  PERSONAL  INFORMATION 

NAME:  _  FACILITY: _ 

*ATC  EXPERIENCE: 

(REPORT  ALL  RELEVANT  EXPERIENCE;  CIRCLE  YOUR  CURRENT  POSITION) 

_ FPL  CONTROLLER  _  YEARS 

_ FIRST  LEVEL  SUPERVISOR  _  YEARS 

_ OTHER  (SPECIFY)  _ 

*  PI  LOT  EXPERIENCE  _ NO  _ YES  _ YEARS 

***************************************************************** 

3  OUTLINE 

*  OPERATIONAL  PROCEDURES 

*  DISPLAYS 

*  IN PUTS/ CONTROLS 

**r*r*******;*t***:*c*r*r*r**c*r*r**:**f*:****:**r*r*:*r4r***4r**r***r**********t4r4r*:**r* 

4  OPERATIONAL  PROCEDURES 

♦COMMENTS/RESERVATIONS  CONCERNING  USE  OF  THE  DATA  LINK  ATC  SERVICES 
IN  A  FULL  DATA  LINK  EQUIPAGE  ENVIRONMENT: 

ALL  SERVICES _ 

TRANSFER  OF  COMMUNICATION 

ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 
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*************** 

5  OPERATIONAL  PROCEDURES  (CONTINUED) 

* COMMENTS/ RESERVATIONS  CONCERNING  USE  OF  DATA  LINK  ATC  SERVICES 
IN  MIXED  EQUIPAGE  ENVIRONMENT 

ALL  SERVICES _ 

TRANSFER  OF  COMMUNICATION 
ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 

**************************************************************** 

6  OPERATIONAL  PROCEDURES  (CONTINUED) 

* COMMENTS/RESERVATIONS  CONCERNING  PILOT  MESSAGE  ACKNOWLEDGEMENT 
AND  CONTROLLER  NOTIFICATION  OF  MESSAGE  STATUS 

ALL  SERVICES 


TRANSFER 

OF  COMMUNICATION 

ALTITUDE 

ASSIGNMENT  (QZ) 

INTERIM 

ALTITUDE  ASSIGNMENT 

(QQ) 

INTERIM 

ALTITUDE  ASSIGNMENT 

(MENU) 

’k'k'k'k'k'k'k'k-k-k'kieie-k-kis'k'k-k-k-k'k-k-k'k'k-k'kie'k'k'kic'k'k’k-k-k-k-kis'k'k-k'k'k'k'kit'kieieicie'kicit'kie'k'kick’k 

7  OPERATIONAL  PROCEDURES  (CONTINUED) 

*COMMENTS/RESERVATIONS  CONCERNING  HANDLING  OF  FAILED  MESSAGES 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION 
ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 
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***************************************************************** 
8  OPERATIONAL  PROCEDURES  (CONTINUED) 

♦COMMENTS/RESERVATIONS  CONCERNING  WORDING/ PHRASEOLOGY  USED  IN 
DATA  LINK  MESSAGES 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION 
ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 


****************************************************************** 
9  OPERATIONAL  PROCEDURES  (CONTINUED) 

*COMMENTS/RESERVATIONS  CONCERNING  TIMING/PACING  OF  COMMUNICATIONS 
IN  A  FULL  DATA  LINK  EQUIPAGE  ENVIRONMENT 

ALL  SERVICES _ _ 


TRANSFER 

OF  COMMUNICATION 

ALTITUDE 

ASSIGNMENT  (QZ) 

INTERIM 

ALTITUDE  ASSIGNMENT 

(QQ) 

INTERIM 

ALTITUDE  ASSIGNMENT 

(MENU) 

***************************************************************** 
10  OPERATIONAL  PROCEDURES  (CONTINUED) 

*COMMENTS/RESERVATIONS  CONCERNING  TIMING/PACING  OF  COMMUNICATIONS 
IN  A  MIXED  EQUIPAGE  ENVIRONMENT 

ALL  SERVICES  _ 


TRANSFER 

OF  COMMUNICATION 

ALTITUDE 

ASSIGNMENT  (QZ) 

INTERIM 

ALTITUDE  ASSIGNMENT 

(QQ) 

INTERIM 

ALTITUDE  ASSIGNMENT 

(MENU) 
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***************************************************************** 
11  OPERATIONAL  PROCEDURES  (CONTINUED) 

♦COMMENTS/RESERVATIONS  CONCERNING  COMPATIBILITY  OF  DATA  LINK 
AND  VOICE  PROCEDURES 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION  ~ 

ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 


***************************************************************** 

12  OPERATIONAL  PROCEDURES  (CONCLUDED) 

* COMMENTS/RESERVATIONS  CONCERNING  (TESTBED)  DESIGN  FEATURES  AND 
PRACTICAL  USE  OF  DATA  LINK  ATC  SERVICES 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION 

ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 

***************************************************************** 

13  DISPLAYS 

*COMMENTS/ RESERVATIONS  CONCERNING  DISPLAY  OF  DATA  LINK  EQUIPAGE 
AND  ELIGIBILITY 

14  DISPLAYS  (CONTINUED) 

*COMMENTS/ RESERVATIONS  CONCERNING  DISPLAY  OF  DATA  LINK  MESSAGE  AND 
TRANSACTION  STATUS  INFORMATION 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION 
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ALTITUDE  ASSIGNMENT  (QZ)  _ 

INTERIM  ALTITUDE  ASSIGNMENT  (QQT 


INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 


***************************************************************** 

15  DISPLAYS  (CONCLUDED) 

* COMMENTS/ RESERVATIONS  CONCERNING  DISPLAY  OF  PRE-BUILT  MENU 
MESSAGES 


***************************************************************** 

16  INPUTS/ CONTROLS 

♦COMMENTS/RESERVATIONS  CONCERNING  MESSAGE  ELIGIBILITY  AND  MESSAGE 
COMPOSITION  RULES 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION 
ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 

17  INPUTS/CONTROLS  (CONTINUED) 

* COMMENTS/ RESERVATIONS  CONCERNING  CAPABILITIES  AND  COMMANDS  TO 
INITIATE  UPLINK  OF  MESSAGE 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION 
ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 
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***************************************************************** 

18  INPUTS/CONTROLS  (CONCLUDED) 

♦COMMENTS/RESERVATIONS  CONCERNING  AUTOMATIC  DELETION  OF  WILCO 
STATUS  AND  UPDATE  OF  NAS  (DATA  BLOCKS  AND/OR  DATA  BASE) 

ALL  SERVICES  _ 

TRANSFER  OF  COMMUNICATION 

ALTITUDE  ASSIGNMENT  (QZ) 

INTERIM  ALTITUDE  ASSIGNMENT  (QQ) 

INTERIM  ALTITUDE  ASSIGNMENT  (MENU) 
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DESCRIPTION 


This  report  documents  the  statistical  treatment  and  results  of  the 
Wrap-Up  Questionnaire  which  was  given  to  the  seven  test  controllers 
during  the  debriefing  after  the  final  test  runs.  A  copy  of  the 
questionnaire  is  attached  herein. 

The  questionnaire  is  comprised  mainly  of  7-level  rating  scales  and 
free  narrative  cciument  questions.  The  numerical  values  ana  verbal 
labels  attached  to  the  7-level  rating  scale  questions  are: 

7  =  Very  Good 
6  =  Good 

5  =  Slightly  Good 
4  =  Fair  (center  scale) 

3  =  Slightly  Poor 
2  =  Poor 
1  =  Very  Poor 

These  values  are  used  for  the  computation  of  the  rating  scale 
statistical  results.  Narrative  comments  for  all  questions  are 
compiled  and  attached  herein.  Also,  a  frequency  count  of 
controllers  making  similar  statements  is  attached. 

RATING  SCALE  ANALYSIS 

Table  E-l  shows  the  resulting  descriptive  and  inferential 
statistics  for  the  17  items  rated  using  the  7-level  rating  scale. 
Means  and  variation  about  the  means  were  computed  for  each  of  the 
items.  Student's  t  scores  were  computed  to  test  if  the  means  were 
significantly  higher  or  lower  than  center  scale  (4) . 

In  table  E-l  the  items  are  presented  in  magnitude  order  of  t  score. 
A  5-percent  confidence  interval  about  each  item  mean  was  also 
computed.  Notes  at  the  bottom  of  the  table  give  a  brief 
explanation  of  the  meaning  of  the  table  headings. 

Comparing  the  t  score  to  the  center  scale  value  of  4  indicates 
whether  that  mean  rating  is  significantly  above  or  below  it.  These 
ranks  are  given  in  the  results  below. 

RATING  SCALE  DATA  RESULTS 

The  following  results,  using  a  95  percent  probability  criterion 
(p=. 05)  for  significance,  were  obtained  for  the  rating  scale 
questions. 

Of  the  17  items  rated,  12  were  rated  GOOD  (significantly  higher 
than  FAIR) ,  5  were  rated  FAIR  (did  not  differ  significantly  from 
FAIR)  and  none  were  rated  POOR  (significantly  lower  than  FAIR)  . 
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TABLE  E-l.  OP  EVAL  WRAP-UP  QUESTIONNAIRE  RATING  SCALE  ANALYSIS 
IN  ORDER  OF  STATISTICAL  SIGNIFICANCE 


TABLE  E-l.  OP  EVAL  WRAP-UP  QUESTIONNAIRE  RATING  SCALE  ANALYSIS 
IN  ORDER  OF  STATISTIC7  SIGNIFICANCE  (Cont'd) 


II  II  II  II 


Cl 

? 

<0 


<0 

d  *f- 
>  >  • 
odd 

25  -o  d 


>•  <  > 


t  x  s 

<o  d  •— 
O  -C  u 
•—  d 


«  d 

« -s 


*! 


5 


>  d 

£  I 

.c 

O)  o 


(A 

X  *-* 
C  (A 

<  d 


£  S 


d 

*_ 

o 


d 

c  ■£ 
<0  _ 


o  = 

H- 

II 


§  £ 
t- 
d 


d 
ip 


o 
• 


W  ||  x 


6  ■ 

QC 


«-  C  H-  — 


5  I 


a.  - 

~  S  £ 

</>  — »  o 

— '  <0  ■— 

d  >  h- 


</> 

»♦- 

°  i 


1  a 
4-* 
-C  * 

o>  J 


c  d 

O  (0 

•—  o 

H-  GO 


d  t_  </> 

o  d 
£*-<*-» 
«  O 


"  £ 
u 


■*-  d 

c  ^ 
o>  *-• 


° 

W'  O  »/^  <J 

o  u  o 


O  ac  C 

0  5  51 


o  d  o  - 


E-3 


Considering  the  nature  of  the  questionnaire  items,  this  shows  a 
high  acceptance  of  the  Data  Link  concepts  evaluated. 

Rank  ordering  the  t  scores  makes  it  possible  to  determine  which 
items  are  notable  as  far  as  degree  of  preference. 

QUESTION-BY-QUESTION  RESULTS 

Question  1  asks  for  a  rating  of  the  simulation  realism.  The 
resultant  rating  is  FAIR. 

Question  2  asks  suggested  improvements  for  realism.  Twelve 
suggestions  are  made.  See  Narrative  Comments  section. 

Question  3  asks  if  training  is  sufficient.  The  response  is 
unanimous,  7  yes  and  0  no.  Significant. 

Question  4  asks  if  the  services  were  used  often  enough  to  give  a 
fair  evaluation.  The  answer  was  significantly  yes  for  each. 


1.  Assigned  Altitude  ("S" ) 

2.  Interim  Altitude  ("S") 

3 .  Interim  Altitude  Menu  Text 

4.  Transfer  of  Communication 

5.  Free  text 


Yes  No 


7 

7 

7 

7 

6 


0 

0 

0 

0 

0 


Question  5  asks  to  document  any  system  malfunctions.  Two  are 
mentioned.  See  Narrative  Comments  section. 

Question  6a  asks  to  evaluate  the  data  input,  DL  /OK  ID. 
Controllers  said  .unanimously  (7-0)  that  it  is  needed,  and  works 
SIGNIFICANTLY  GOOD. 


Question  6b  asks  to  evaluate  the  Data  Link  input,  DL  /OK  S  ID. 
Controllers  said  unanimously  (7-0)  that  it  is  needed,  and  works 
SIGNIFICANTLY  GOOD. 


Question  6c  asks  to  evaluate  the  Data  Link  input,  QQ  R  ID. 
Controllers  said  unanimously  (7-0)  that  it  is  needed,  and  works 
SIGNIFICANTLY  GOOD. 


Question  6d  asks  to  evaluate  the  Data  Link  input,  QQ  Z  ddd  ID. 
Controllers  voted  5  yes,  1  no,  and  1  can't  tell.  It  was  rated 
SIGNIFICANTLY  GOOD  as  to  how  it  works,  but  a  change  is  suggested 
to  eliminate  the  ddd  as  unnecessary. 

Question  7  asks  about  attention  alarms.  See  the  Narrative  Comments 
section. 
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Question  8  asks  to  rate  the  pilot  response  delays.  The  controller 
ratings  were  SIGNIFICANTLY  GOOD. 

Question  9  asks  if  the  extended  pilot  response  delays  of  the 
supplemental  run  were  noticeably  longer.  Yes=2,  No=5. 

Question  10  asks  if  Data  Link  could  be  used  even  with  extended 
pilot  delay.  Yes=6,  No=0,  Can't  Tell=l.  This  is  significant. 

Question  11  asks  if  the  PVD  were  preferred  displayed  or  suppressed. 
Displayed=6,  Suppressed=l . 

Question  12  asks  what  control  events  are  best  for  free  text.  See 
the  Narrative  Comments  section. 

Question  13  asks  for  foreseen  problems  for  aircraft  transitioning 
in  and  out  of  DL  coverage  sectors.  See  Narrative  Comments  section. 

Question  14  asks  if  the  controller  should  have  the  capability  to 
modify  the  Interim  Altitude  menu  text.  Yes=5,  No=l,  Can't  Tell=l. 

Question  15  asks  if  there  is  a  need  to  string  messages  together  for 
one  send.  Yes=4,  No=2 . 

Question  16  asks  how  best  to  design  menu  text. 

Separate  menus  per  service  =  1 
One  menu  allowing  stringing  =  3 
A  separate  free  text  menu  =  2 

Question  17  asks  the  projected  effect  of  Data  Link  on  11  items. 
Seven  items  were  rated  significantly  GOOD,  four  FAIR,  and  none 
SIGNIFICANTLY  POOR.  See  table  E-l  for  items  17a  to  17k. 

Question  18  asks  if  during  a  handoff  the  receiving  controller 
should  also  see  the  transfer  of  comm,  up  arrow.  Yes=6,  No=0,  Can't 
Tell=l.  Significantly  approved. 

Question  19  asks  if  Data  Link  is  effective  when  things  get  busy. 
Three  said  more  effective,  one  said  less  effective,  one  said 
equally,  and  two  can't  determine. 

Question  20  asks  if  the  pilot  need  check  in  by  voice  on  the  new 
frequency  with  Data  Link.  Yes=l,  No=5,  Can't  Tell=l. 

Question  21  asks  if  individual  transfer  of  comm,  messages  can  be 
selected  as  automatic  such  that  the  change  frequency  message  is 
uplinked  upon  handoff  acceptance.  Yes=5,  No=l,  Can't  Tell=l. 

Question  22  asks  for  gaps  in  the  system.  See  Narrative  Comments 
section. 
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Question 

section. 

23 

asks 

what '  s 

liked 

best. 

See 

Narrative 

Comments 

Question 

24 

asks 

what ' s 

liked 

least. 

See 

Narrative 

Comments 

section. 

Question  25  asks  what  future  services  should  Data  Link  do.  See 
Narrative  Comments  section. 


At  the  end  of  the  data  runs  for  scenarios  1  and  2,  controllers  were 
asked  to  rate  the  traffic  size  per  scenario  on  a  scale  from  1  to 
10.  This  form  is  attached  with  the  Wrap-Up  Questionnaire.  Also 
asked  was  the  projected  traffic  rating  which  would  require  D  side 
assistance.  The  results  are  below. 

Traffic  size  Need  D  side 


Scenario  1 
Scenario  2 


Avg  Range 

4  2-6 

4.25  1-6 


Avg  Range 

5.4  4-6 

5.4  4-6 


Interestingly,  the  controllers  rated  the  traffic  size  as  light- 
to-moderate  (4)  for  both  scenarios,  yet  rated  the  level  where  a  D 
side  should  assist  at  5.5  (moderate-to-heavy) ,  only  1.5  points 
higher.  Since  all  the  controllers  projected  the  need  for  a  D  side 
at  6,  it's  assumed  that  5  (moderate  traffic)  is  full  capacity  for 
one  person  at  the  sector.  Thus,  these  ratings  reflect  traffic  size 
in  terms  of  a  two-controller  operation  in  the  field,  and  not 
traffic  size  for  the  controller  himself/herself  in  the  simulation. 

NARRATIVE  COMMENTS 


The  complete  compilation  of  narrative  comments  to  the  Wrap-Up 
Questionnaire  is  attached  to  this  appendix.  The  comments  that 
might  be  interpreted  as  a  consensus  are  summarized  below. 

REALISM.  Several  items  were  mentioned  that  could  increase  realism: 
the  inclusion  of  flight  strips,  increased  and  more  varied  traffic 
patterns,  VOR's,  radials,  airways,  more  typical  speeds,  and  turns. 

ATTENTION  ALARMS.  Four  controllers  said  no  need  for  aural  alarms. 
Three  said  FAILS  should  blink  in  full  data  block;  two  suggested 
something  else  would  be  better  because  blink  is  too  similar  to 
CONFLICT  ALERT. 


RESPONSE  DELAY.  Four  responded  that  they  became  accustomed  to  the 
delay  since  they  would  only  use  Data  Link  for  messages  where  the 
delay  would  not  be  a  problem. 

MULTIMESSAGE.  Four  commented  that  the  capability  to  string 
messages  together  for  one  send  would  be  good  if  it  does  not 
introduce  entry  errors. 


E-6 


AUTO  TOC.  Four  commented  that  an  option  to  make  transfer  of 
communication  automatic  on  an  individual  aircraft  basis  was  a  good 
idea. 

GAPS  IN  THE  SYSTEM.  Five  commented  on  envisioned  "gaps"  in  the 
system,  each  one  of  which  was  different. 

SUMMARY  OF  RESULTS 

The  seven  en  route  controllers  reported  that  the  Data  Link  Op  Eval 
had  fair  realism,  training  was  adequate,  and  malfunctions  minimal. 
They  judged  that  they  had  sufficient  use  of  the  Data  Link  services 
to  give  a  fair  evaluation.  Response  delays  indigenous  do  Data  Link 
transmission  were  judged  no  problem. 

New  system  entries,  "DL  /OK  ID,"  "DL  /OK  S  ID,  "QQ  R  ID,"  and  "QQ 
Z  ddd  ID"  were  judged  significantly  good,  with  a  change  recommended 
for  Z  to  eliminate  ddd.  Menu  text  should  have  the  capability  of 
linking  and  modifying  messages.  No  aural  alarms  were  considered 
necessary  for  attention  alarms.  The  most  prevalent  philosophy  on 
alarms  for  failures  was  to  blink  it. 

The  projected  effects  of  using  the  Data  Link  for  ATC  in  the  field 
were,  for  the  most  part,  SIGNIFICANTLY  GOOD.  Four  of  the  12 
effects  were  rated  FAIR,  and  8  were  rated  SIGNIFICANTLY  GOOD 
including:  system  capacity,  overall  safety,  efficiency,  and 

controller  workload.  Good  effects  were  also  projected  for  sector 
interphone,  R  and  D  controller  coordination,  and  pilot/controller 
communications . 

Of  the  narrative  answers  in  the  comment  section,  the  responses 
amounted  to  a  majority  for  four  questions; 

1.  Realism  can  be  improved  for  the  simulation. 

2.  No  aural  alarms. 

3.  Response  delay  is  acceptable. 

4.  Multimessaging  is  good  menu  text  item. 

5.  Auto  TOC  is  a  good  feature. 

Many  ideas  of  future  services  are  listed.  A  listing  of  narrative 
comments  per  question  in  the  Wrap-Up  Questionnaire  follows. 
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NARRATIVE  RESPONSES  TO  EACH  QUESTION 
WRAP-UP  QUESTIONNAIRE 

QUESTION  1.  No  comments 

QUESTION  2.  How  can  we  enhance  realism? 

CONTROLLER 

1  The  design  of  the  simulation  is  as  good  as  it  could  be 
considering  the  limitation  of  the  lab  environment. 

2  Maybe  use  an  Aeronautical  Center  sector  setup-  Handoff 
the  planes  to  us  vs.  their  appearance.  Allow  the  remotes 
to  have  quicker  access  of  assigned  altitudes. 

3  More  aircraft. 

4  Vary  the  traffic  patterns  a  little  more,  use  better 
trained  pilots. 

5  Aircraft  realistically  would  turn  slower,  pilots  would 

read  back  full  clearances  and  answer  up  quicker. 

6  Have  the  Mode  C  work  on  all  aircraft,  don't  have  the 
aircraft  "appear"  so  close  to  the  traffic.  Have  flight 
process  strips. 

7  The  lack  of  VOR's,  radials,  airways  in  UDS  Center  did 

have  a  limiting  effect  on  realism.  Also,  speeds  of  190- 
240  kt  on  jets  is  unrealistic.  SIMOP's  should  be  briefed 
on  realistic  speeds  and  begin  the  simulations 

accordingly. 

QUESTION  3.  Any  ideas  as  to  other  training  that  would  have  aided 
you? 

CONTROLLER 

7  I  felt  I  needed  a  more  complete  briefing  on  procedures 
and  potential  of  both  the  GAT  and  the  NASA/AMES 
simulator. 

QUESTION  4.  Did  you  use  the  services  in  this  test  often  enough 
to  be  able  to  give  a  fair  evaluation? 

CONTROLLER 

5  Good  services. 
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QUESTION  4  (Continued) 

CONTROLLER 

7  I  feel  F/T  needs  additional  development.  Most  types  of 
F/T  messages  will  probably  be  composed  and  delivered  by 
D-side. 

QUESTION  5.  Please  document  any  inadvertent  malfunctions  that, 
working  properly,  would  have  helped  you  make  a 
fairer  evaluation. 

CONTROLLER 

1  The  D/L  functions  worked  consistently  enough. 

2  Lack  of  Mode  C  in  scenario  1  made  it  too  busy  to  try  to 
evaluate  (until  we  simply  manually  reported  the  altitudes 
w/o  asking  remotes) . 

4  None. 

6  The  items  listed  in  #2. 

7  None. 

QUESTION  6a.  Please  evaluate  the  following  Data  Link  inputs: 

"DL  /OK  ID,”  Controller  with  track  control 
establishes  DL  comm,  with  a/c  (a/c  is  already  on 
frequency)  . 

CONTROLLER 

1  Works  good  but  procedures  (like  for  track  control)  would 
be  needed  to  prevent  someone  inadvertently  using  this. 

7  But  still  requires  manual  coordination  with  transferring 

controller. 

QUESTION  6b.  Please  evaluate  the  following  data  link  inputs: 

"DL  /OK  S  ID,"  controller  with  track  control 
establishes  DL  comm,  and  sends  his  frequency  to  a/c. 

CONTROLLER 

1  Better  than  DL  /OK  ID  because  it  also  sends  the  freq. 

2  Must  define  procedure  because  I  don't  want  somebody 
taking  communications  of  a/c  I  might  need  to  talk  to. 

7  Still  requires  manual  coordination  with  transferring 

controller. 
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QUESTION  6c .  Please  evaluate  the  following  data  link  inputs: 

"QQ  R  ID,"  controller  with  track  control  and  DL 
comm,  uplinks  the  requested  altitude  to  the  a/c. 

CONTROLLER 

1  Real  good,  because  it  minimizes  number  of  keys  to  push. 

7  Very  useful  service  for  high  altitude  controller. 

QUESTION  6d.  Please  evaluate  the  following  data  link  inputs: 

"QQ  Z  ddd  ID,"  initiating  controller  send  the  same 
Int.  Alt.  to  the  pilot  that  the  receiving  controller 
has  already  entered  in  the  FDB. 

CONTROLLER 


1  Don't  know  if  this  is  necessary  or  if  procedures  can  be 
written  around  it. 


2  I  sent  an  incorrect  altitude  to  aircraft  (intentionally). 
After  the  initial  acceptance  by  the  a/c,  there  was  no 
indication  that  this  a/c  received  and  is  climbing  to  the 
wrong  altitude. 

3  Specific  sites  only. 

7  This  procedure  is  not  routinely  used  in  many  facilities, 
however,  after  the  proper  coordination  it  can  still  be 
a  useful  procedures. 


QUESTION  7.  How  were  the  attention  alarms?  Should  anything  be 
made  to  blink  or  not  blink?  How  about  aural  alarms? 


CONTROLLER 


1  No  aural  alarms  needed.  Fail  in  data  block  is  good 
enough. 

2  No  aural  alarms.  An  overambitious  conflict  alert  in  the 
facility  is  already  making  us  complacent  to  blinking. 
Maybe  a  character  in  the  altitude  field  indicating  a  need 
for  action  would  be  sufficient. 

3  No  aural  alarms. 

4  OK,  but  the  fail  on  an  altitude  should  flash,  and  an 
unsuccessful  transfer  of  communication  should  flash  in 
hour  glass.  There  should  also  be  a  distinction  between 
FAIL  and  when  a  pilot  hits  the  UNABLE  button. 
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QUESTION  7  (Continued) 

CONTROLLER 

5  No  blinking.  Blinking  reminds  a  controller  of  conflict 
alert  warning.  No  aural  alarms.  There  is  enough  noise 
already  in  the  control  room. 

6  If  something  fails  I  think  the  entire  data  block  should 
blink  to  make  you  more  aware  of  the  failure. 

7  In  TOC,  I  feel  the  receiving  controller  needs  an 
attention  device  when  hourglass  is  displayed.  Perhaps 
either  highlight  (double  intensity)  or  perhaps  even  flash 
the  hour  glass  for  a  few  seconds  parameter.  Without  the 
attention  device,  I  found  my  normal  scan  being 
interrupted  to  check  for  eligibility. 

QUESTION  8.  How  do  you  rate  the  pilot  response  delays? 

CONTROLLER 

1  If  something  immediate  is  needed  to  be  done,  voice  will 
have  to  be  used  anyway,  so  even  though  there  is  delays 
it  isn't  a  real  major  problem. 

2  I  got  accustomed  to  it. 

5  Data  Link  delays  were  good.  Voice  delays  were  not 
realistic,  they  took  too  long. 

7  The  response  time  was  not  significantly  different  than 
what  we  see  today  and  did  not  cause  a  problem.  However, 
I  believe  this  is  a  critical  issue  that  must  be 
successfully  resolved  in  system  architecture  prior  to 
implementation.  Response  times  greater  than  12-20 
seconds  would,  in  my  judgement,  be  excessive  for  some  ATC 
SERVICES  in  certain  time-critical  situations.  Or, 
perhaps  voice  will  continue  to  be  the  choice  of 
controllers  in  those  situations. 

QUESTION  9.  For  the  run  with  extended  pilot  response  delay,  were 
the  delays  noticeably  longer? 

CONTROLLER 

1  Extended  delays  were  not  any  worse  since  there's  a  delay 
with  D/L  anyway. 

2  Unless  the  delays  become  unreasonable  I  don't  think 
they're  too  difficult  to  compensate  for. 
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QUESTION  9  (Continued) 

CONTROLLER 

5  The  longer  delays  would  not  be  helpful  when  busy. 

7  Only  in  two  or  three  instances  did  I  notice  a  significant 
difference. 

QUESTION  10.  Could  Data  Link  still  be  utilized  effectively  even 
with  the  extended  pilot  response  delay? 

CONTROLLER 

1  It  would  still  work  good  for  transfer  of  communication 
and  most  altitude  changes. 

2  See  9 . 

5  Only  usable  with  transfer  of  communication;  for  altitudes 
changes,  the  delay  is  too  long. 

7  As  it  was  experienced  in  the  lab  -  yes,  but  see  question 

8. 

QUESTION  11.  Did  you  prefer  the  PVD  status  list  displayed  or 
suppressed? 

CONTROLLER 

1  Don't  need  it  if  status  shows  up  in  data  block  in  some 
form.  The  list  takes  too  much  time  to  look  through. 

2  I  rarely  used  it  anyway. 

4  Needed  for  free  text  messages. 

5  I  could  see  responses  to  my  messages  at  a  glance. 

7  I  believe  the  status  list  is  essential  and  should  be 

displayed.  However,  it  continues  to  need  refinement  to 
make  it  an  effective  tool.  First,  I  feel  it  is  not 
necessary  to  display  "SENT”  and  "HOLD”  on  the  receiving 
controller  PVD  during  a  TOC.  Secondly,  we  need  to 
continue  to  explore  ways  to  "display  the  exception," 
i.e.,  failures,  rather  than  display  every  "SENT."  Also, 
my  thinking  at  this  time  is  that  FT  will  become  primarily 
a  D-side  function,  therefore,  display  FT  acknowledge  in 
D-side  RVD.  In  short,  keep  the  STATUS  LIST  -  but  edit 
it  substantially  to  keep  in  USABLE  by  the  controller. 
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QUESTION  12.  What  control  events  do  you  foresee  would  be  best 
handled  by  free  text? 

CONTROLLER 

1  Only  during  emergencies  or  when  freq.  ties  are  lost.  I 
don't  like  it  for  control  functions. 

2  Local  ride  reports,  miscellaneous  messages. 

3  Route  changes. 

4  Vectoring  and  speed  adjustments  minor  route  amendment. 

5  Send  messages  from  company  or  ARINC,  send  WX  sequences, 
etc. 

6  A  freq.  change  during  stuck  mick  times. 

7  (1)  Field  10  amendments,  that  is,  route  revisions.  (2) 
Current  ride  reports  when  not  included  in  weather 
products.  (3)  Speed  assignments.  (4)  Very  limited  use  as 
a  radar  vector  tool.  Generally  speaking,  I  found  FT  to 
be  marginally  useful  to  the  "R"  controller  during  slow 
periods,  and  not  at  all  useful  during  busy  periods.  The 
"R"  controller  simply  does  not  have  the  time  to  make 
large  numbers  of  KEYSTROKES. 

QUESTION  13.  What  kind  of  problems  are  foreseen  when  aircraft 
transition  in  and  out  of  DL  coverage  sectors? 

CONTROLLER 

1  Controllers  transitioning  between  use  of  D/L  and  freq. 
It  might  cause  some  confusion  unless  the  areas  of  lack 
of  coverage  were  well  known. 

2  As  long  as  DL  eligibility  can  be  /OK'd,  none  at  all. 

3  This  should  not  be  a  problem.  Either  the  controller  does 
or  does  not  have  Data  Link. 

4  Lost  aircraft  communication,  if  only  temporary  causing 
the  DL  controller  having  to  enter  extra  messages  or  make 
additional  voice  transmission. 

5  Can't  envision  any  right  now. 

6  The  pilot  attention  is  the  weak  link  and/or  the  placement 
of  the  Data  Link  device. 
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QUESTION  13  (Continued) 

CONTROLLER 

7  It  is  difficult  to  answer  this  question  without  more 
information  about  the  coverage  capability  of  the  Mode  S 
sensors.  However,  it  is  a  crucial  question  that  will 
need  to  continue  to  be  looked  at. 

QUESTION  14.  Should  the  controller  have  the  capability  to  modify 
the  interim  altitude  menu  text? 

CONTROLLER 

1  They  should  be  set  by  facility  procedure  if  an  individual 
controller  has  different  preferences  he  can  always  do  it 
another  way. 

2  Give  us  the  chance  to  issue  the  particular  clearances  we 
like  to  use. 

4  Not  really  important  because  the  controller  will  know 
which  items  he  will  be  using  and  probably  won't  have  it 
displayed.  Also,  I  suggest  that  the  menu  text  and  the 
status  list  be  put  on  different  keys. 

5  Different  controllers  have  different  restrictions  that 
they  use,  they  should  have  the  ability  to  use  the 
restrictions  that  they  want. 

7  In  other  words,  modify  DDD  to  120,  230,  etc.  However, 
I  foresee  most  menu  text  items  to  be  predetermined  by 
sector  and  specified  by  Procedures  Facility  Directives. 

QUESTION  15.  Is  there  need  to  string  messages  together  with  one 
send  (i.e.,  (1)  fix  cross  alt.  (2)  speed,  (3) 
heading  =  DL  1  2  3  FLID  enter) ? 

CONTROLLER 

1  That  would  save  a  lot  of  time  for  both  pilot  and 
controller  as  long  as  the  individual  items  were  clearly 
identified  in  a  format  the  pilot  would  notice. 

2  I  believe  tnat  one  clearance  per  transmission  sets  a 
better  rhythm  and  reduces  chances  of  pilot  error. 

4  Absolutely. 

5  During  an  arrival  sequence,  these  three  items  are  given 
together  frequently. 
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QUESTION  15  (Continued) 

CONTROLLER 

7  The  need  is  there,  and  it  should  be  available  as  a 
controller  tool  just  as  it  is  today.  Naturally,  it  will 
be  applied  only  in  carefully  controlled  areas. 

QUESTION  16.  How  best  to  design  menu  text? 

Separate  menus  (AA,  IA,  speeds,  headings,  free 
text)?  One  menu  for  string  Xing,  IA  speeds, 
headings)?  A  free  text  menu  (of  typical  free  text 
messages) ? 

CONTROLLER 

1  One  menu  for  multiple  messages  is  needed. 

2  Try  some  different  ways. 

4  Stringing  is  an  excellent  idea. 

5  Need  to  be  able  to  put  in  whatever  restrictions  are 
needed,  within  a  defined  order.  Sometimes  IA,  speeds  may 
need  to  be  sent  without  headings,  other  times  all  three 
need  to  be  sent. 

7  I  continue  to  believe  it  will  be  a  very  valuable  tool  for 
the  controller;  however,  substantial  design  work  remains 
to  be  done  to  maximize  its  utility  to  the  controller. 
Some  of  this  may  be  facility  or  even  sector  specific. 
One  principle  that  should  be  further  incorporated  in  it 
is  to  minimize  the  possibility  of  inadvertently  selecting 
the  wrong  MT  message.  Instead  of  naming  them  simple  A- 
Z,  perhaps  identify  the  message  by  a  Letter/Number/Name 
that  actually  bears  a  resemblance  to  the  message.  As  far 
as  stringing  together  vs.  individual  menus,  I  feel  we 
need  to  look  at  those  options  in  a  test  bed  situation 
before  I  can  make  an  informed  judgment.  Also,  given  the 
increased  capacity  of  ISSS  over  the  PVD,  the  answer  would 
be  different. 

QUESTION  17.  For  the  Data  Link  Services  tested,  what  would  be  the 

projected  effect  of  Data  Link  at  your  position  on 
the  following: 

CONTROLLER 

1  Safety  may  increase  due  to  reduced  workload. 
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QUESTION  17  (Continued) 

CONTROLLER 

2  I  found  that  when  I  was  issuing  verbal  clearances  my 
mouth  was  busy,  and  when  I  was  issuing  D/L  clearances  my 
hands  were  very  busy. 

4  On  frequency  changes. 

5  When  using  Data  Link  mostly  and  not  much  talking  it 
tended  to  lull  me  into  a  relaxed  mode  that  lowered  my 
attention  level. 

7  At  this  stage  I  would  have  to  qualify  most  answers  with 
"It  depends." 

QUESTION  18.  During  a  handoff,  should  the  receiving  controller 
also  see  the  up  arrow  in  the  FDB  when  a  TOC  is  sent 
to  the  pilot? 

CONTROLLER 

1  Not  a  need  to  know,  but  extremely  helpful. 

QUESTION  19.  When  things  get  busy  is  Data  Link 
More  effective 
Less  effective 
Equally  effective 
Can't  determine? 

CONTROLLER 

1  Only  if  a  D-side  or  coordinator  is  involved.  With  only 
one  person  working  it  isn't  anymore  effective  because  you 
still  have  to  do  everything. 

2  With  more  practice  I  think  I  (or  two  people  working  a 
sector)  could  issue  two  clearances  simultaneously. 

5  Especially  helpful  is  transfer  of  communications 
function. 

6  My  answer  may  change  if  these  task  could  be  done  by  your 
D-side,  but  we  did  not  get  to  experience  test. 

7  In  the  test  environment  -  I  found  that  at  20%  equipage 
and  busy  session  I  did  not  use  Data  Link.  At  80%,  it  was 
feasible  to  use  it  more.  However,  some  of  the  controller 
workload  may  be  shared  with  D-  and/or  H/O  controller 
through  proper  procedures. 


E-16 


QUESTION  20.  Since  the  pilot  has  the  new  frequency  on  display  for 

reference,  does  he  need  to  voice  check  in  on  the  new 
frequency? 

CONTROLLER 

1  Even  if  he  doesn’t  come  over,  with  D/L  you  can  always  go 
get  him  again. 

2  As  long  as  he  is  receiving  the  correct  info. 

4  But  there  needs  to  be  some  way  of  insuring  that  he 
switches  to  the  correct  freq. 

5  But  we  need  to  make  sure  that  pilots  switch  over  when 
given  a  new  frequency,  maybe  the  Data  Link  could  be  tied 
into  the  pilots  actually  switching  the  numbers  on  their 
radio. 

6  I  personally  like  the  verbal  enforcement. 

7  I  feel  that  during  field  implementation,  a  voice  check 
is  still  desirable. 

QUESTION  21.  Is  there  a  need  for  selectable  automatic  TOC  (i.e., 
change  frequency  message  is  uplinked  when  receiving 
controller  accepts  handoff)  on  a  per  aircraft  basis? 
Possible  entry  could  be  adding  "S"  to  handoff 
initiation. 

CONTROLLER 

1  That  could  be  useful  in  some  sectors. 

2  Anything  to  save  an  entry  -  make  indication  in  data 
block. 

4  Excellent  idea. 

5  That  would  be  helpful  as  long  as  it  is  only  on  a  per 
aircraft  basis,  many  times  aircraft  are  handed  off  prior 
to  the  time  that  they  should  be  shipped. 

QUESTION  22.  Using  Data  Link  should  leave  no  ’’gaps"  in  the 

system.  Did  you  see  any  places  where  gaps  in 
positive  control  could  occur?  If  so,  any  ideas  as 
how  to  prevent  them? 

CONTROLLER 

1  Holes  in  coverage  if  they  are  not  well  known  could  be  a 
serious  problem. 
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QUESTION  22  (Continued) 

CONTROLLER 

2  If  the  pilots  cleared  altitude  is  different  than  that 
reflected  in  the  data  block,  then  let's  see  it.  (i.e., 
"280T263"  Time  Share  sent  altitude  to  show  "290S263." 

3  No.  Positive  control  is  maintained  via  radio. 

4  Yes.  When  an  interim  altitude  not  sent  to  the  pilot 
overrides  the  actual  altitude  sent  previously  that 
failed.  A  failed  altitude  should  always  have  a  flashing 
FAIL  time  share  with  the  altitude  that  failed  and  maybe 
even  the  previous  assigned  altitude.  Must  remember  that 
a  failed  altitude  must  still  be  protected  for. 

5  Even  with  Data  Link,  pilots  can  comply  in  a  timely  manner 
or  not;  their  choice.  I  cannot  envision  how  to  correct 
this  "gap." 

6  No. 

7  I  believe  a  TOC  without  a  voice  check-in  on  the  freq. 
does  leave  a  gap  during  entire  stages  of  implementation. 
I  believe  voice  check-in  is  desirable. 

QUESTION  23.  What  about  Data  Link  do  you  like  best,  and  why? 

CONTROLLER 

1  If  another  person  is  doing  some  of  the  inputs,  it  leaves 
more  time  for  the  radar  control  to  handle  traffic 
situations. 

2  The  ability  to  transfer  communications.  The  pilot  being 
able  to  read  the  frequency  greatly  reduces  the  chances 
for  listening  errors  often  encountered  when  transferring 
communications . 

3  It  has  the  capability  of  increasing  the  number  of 
aircraft  that  a  given  sector  can  control. 

4  Don't  know  what  I  like  best .  but  not  having  to  deal  with 
incorrect  pilot  readbacks  has  to  be  the  biggest 
advantage. 

5  I  like  the  ability  to  send  messages  to  pilots  even  if  the 
frequencies  go  out,  or  there  is  a  stuck  mike  or  they  are 
just  not  answering  by  voice.  It  would  be  a  good  backup 
so  we  can  stay  in  contact  almost  100%  of  the  time. 
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QUESTION  23  (Continued) 

CONTROLLER 

6  I  like  the  "ability"  to  use  it  when  I  decide.  It  is  a 
tool  that  with  experimentation  each  controller  will 
determine  what  function  he/ she  feels  comfortable  using. 
The  stuck  mike  scenario  Data  Link  would  be  a  blessing. 

7  a.  The  ability  to  free-up  voice  channel  by  uplinking 
routine  and  repetitive  clearances  (and  weather  produces)  . 

b.  An  additional  backup/ emergency  communications 

channel . 

c.  The  potential  to  share  more  of  workload  to  D- 
controller  and/or  Handoff  Controller. 

QUESTION  24.  What  about  Data  Link  do  you  like  least,  and  why? 

CONTROLLER 

1  Possibility  of  coverage  holes. 

2  No  readbacks  and  the  lack  of  ability  to  double  check 
assigned  altitudes  of  transitioning  aircraft  -  (Maybe 
they  should  check  on  while  a/c  in  level  flight  remain 
silent) . 

3  It  will  take  to  long  to  implement. 

4  The  Hourglass  symbology  in  front  of  the  call  sign  has  to 
go.  It  is  too  close  to  the  first  letter  of  other  call 
sign  and  causes  confusion  between  our  carrier 
designations  such  as  AL,  XL,  XAL,  XEAL,  XAAL. 

5  In  the  future  when  a  majority  of  the  a/c  have  it  and  we 
are  using  it  a  lot,  I  think  it  will  be  more  of  a  problem 
staying  alert. 

7  a.  During  heavy  traffic,  the  switching  back  and  forth 
from  voice  to  D/L  makes  it  difficult  to  maintain  the 
"rhythm"  that  is  required  to  work  heavy  traffic, 
b.  Loss  of  "personal  contact"  with  pilots. 

QUESTION  25.  What  future  services  would  you  like  to  see  Data  Link 
do? 

CONTROLLER 

1  Automatic  altimeter  uplinks  (controller  out  of  the  loop, 
so  he  doesn’t  even  have  to  think  about  the  altimeters). 
Menu  text  multiple  message  uplinks  (not  in  free  text 
though)  such  as  speed  control,  heading. 
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QUESTION  25  (Continued) 

CONTROLLER 

2  Assign  headings  -  (i.e.,  QAK  or  DL  H  dd  CID)  (dd  =  Heading 
divided  by  10),  i.e.,  34  =  340 

Assign  speeds  -  (i.e.,  QAK  or  DL  S  dd  CID)  (dd=Airspeed 
divided  by  10),  i.e.,  25  =  250 

Assign  Routes  -  (i.e.,  Tie  in  the  Data  Link  with  my  route 
QAK  similar  to  the  way  you  have  done  with  the  altitude. 
Expand  the  use  of  the  route  key  to  allow  (a)  entering  a 
heading  to  intercept  an  airway,  (b)  the  airways,  (c) 
enter  a  previously  defined  route  stored  in  a  database. 
Link  the  Data  Link  with  extended  flight  management 
database  systems  in  the  cockpit. 

3  Weather  and  other  messages  that  are  workload  on  the 
controller  and  not  specifically  control  related. 

4  Provide  for  two-way  communication  with  the  pilot, 
allowing  the  pilot  to  send  lengthy  route  change  request 
would  be  a  great  advantage. 

5  I  would  like  to  see  NOTAMS,  SIGMETS,  center  WX 
advisories,  etc.,  issued  via  Data  Link.  I  would  like  a 
capability  to  send  these  message  and  others  composed  via 
free  text  to  all  airplanes  on  my  frequency  with  one 
message.  I  would  like  to  be  able  to  send  routings  up  to 
aircraft  via  Data  Link  at  the  same  time  it  is  being 
updated  in  the  computer.  Maybe  we  could  also  find  a  way 
to  issue  holding  instructions  via  Data  Link.  I  think  it 
would  be  easier  for  pilots  to  read  long,  and,  complicated 
instructions  than  to  hear  them  over,  and  over,  and  over 
sometimes  on  lousy  frequencies. 

6  Automatically  transmit  latest  ATIS,  SIGMETS,  weather 
advisories.  The  menu  text  should  be  open  for  changes  or 
additions  by  the  controller  at  all  times.  As  things  in 
the  sector  change,  i.e.,  if  the  menu  text  is  pre- 
established  or  supervisor  established  and  you  would  go 
into  unforeseen  holding  you  could  not  Data  Link  the 
holding  instructions,  which  could  really  help  and  may  be 
the  difference  between  going  under  and  not. 

7  I  believe  the  next  step  is  to  develop  D/L  products  that 
can  be  used  by  G/A  and  other  VFR  pilots,  such  as  exchange 
of  traffic  information,  airspace  information,  D/L  weather 
products. 
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ATTACHMENT 


Name 

Data  Link 

Evaluation 

Date 

Sector 

FAA  Proj . 
4-18-89 

No.  T2001B 

WRAP-UP  QUESTIONNAIRE 


1 .  How  would 

you  rate 

the  simulation 

realism? 

(Circle 

One) 

Note : 

The  abbreviations  below 

will  be 

used  later 

on. 

Consider  "F"  as 

borderline  (i.e 

. ,  maybe) 

acceptable. 

Very 

Slightly 

Slightly 

Very 

Can't 

Good 

Good 

Good 

Fair 

Poor 

Poor 

Poor 

Tell 

(VG) 

(G) 

(SG) 

(F) 

(SP) 

(P) 

(VP) 

(CT) 

2.  How 

can  we 

enhance 

realism? 

(Use  extra  space 

:  on  back 

•) 

3.  Was  training  sufficient?  Yes _ ,  No _ .  Any  ideas  as  to  other 

training  that  would  have  aided  you? 


4.  Did  you  use  the  services  in  this  test  often  enough  to  be  able 
to  give  a  fair  evaluation?  (Circle  one,  CT  =  can't  tell) 


1. 

Assigned  Altitude  ("S") 

yes 

no 

CT 

2. 

Interim  Altitude  ("S") 

yes 

no 

CT 

3. 

Interim  Altitude  Menu  Text 

yes 

no 

CT 

4. 

Transfer  of  Communication 

yes 

no 

CT 

5. 

Free  Text 

yes 

no 

CT 

Comment? 


5.  Please  document  any  inadvertent  malfunctions  that,  working 
properly,  would  have  helped  you  make  a  fairer  evaluation. 


6.  Please  evaluate  the  following  Data  Link  inputs  especially 
encountered  in  the  last  run.  Circle  the  rating. 

a.  DL  /OK  ID  Controller  with  track  control  establishes  DL 
comm,  with  a/c.  (a/c  is  already  on  frequency.) 

Is  this  needed?  yes _  no _  can't  tell _ 

How  well  does  it  work?  VG  G  SG  F  SP  P  VP  CT 

Comment? 


b.  DL  /OK  S  ID  Controller  with  track  control  establishes  DL 
comm,  and  sends  his  frequency  to  a/c. 

Is  this  needed?  yes _  no _  can't  tell _ 

How  well  does  it  work?  VG  G  SG  F  SP  P  VP  CT 

Comment? 


c.  QQ  R  ddd  ID  Controller  with  track  control  and  DL  comm, 
uplinks  the  requested  altitude  to  the  a/c. 

Is  this  needed?  yes _  no _  can't  tell _ 

How  well  does  it  work?  VG  G  SG  F  SP  P  VP  CT 

Comment? 


d.  QQ  Z  ddd  ID  Initiating  controller  sends  the  same  Int.  Alt. 
to  the  pilot  that  the  receiving  controller  has 
already  entered  in  the  FDB. 

Is  this  needed?  yes _  no _  can't  tell _ 

How  well  does  it  work?  VG  G  SG  F  SP  P  VP  CT 
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Comment? 


7.  How  were  the  attention  alarms?  Should  anything  be  made  to 
blink  or  not  blink?  How  about  aural  alarms? 


8.  How  do  you  rate  the  pilot  response  delays?  Disregard  the 
second-to-last  run  (which  had  extended  delays) . 

Circle  one  (see  l.)s  VG  G  SG  F  SP  P  VP  CT 

Comment? _ _  _ 


9.  For  the  second-to-last  run  (with  extended  pilot  response 
delay) ,  were  the  delays  noticeably  longer?  yes _  no _ . 

Comment?  _ _ _ _  _ 


10.  Could  Data  Link  still  be  utilized  effectively  even  with  the 
extended  pilot  response  delay?  yes _  no _  can't  tell _ 

Comment?  _ 


11.  Did  you  prefer  the  PVD  status  list  displayed  or  suppressed? 
(Check  one)  Displayed _  Suppressed _ 

Comment?  _  _  _  _ 


12 .  What  control  events  do  you  foresee  would  be  best  handled  by 
free  text? 


13.  What  kind  of  problems  are  foreseen  when  aircraft  transition 
in  and  out  of  DL  coverage  sectors? 
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14.  Should  the  controller  have  the  capability  to  modify  the 
interim  altitude  menu  text?  yes _  no _  can’t  tell _ 

Comment?  _ _ _ 


15.  Is  there  need  to  string  messages  together  with  one  send  (i.e., 
(l)Fix  cross  alt,  (2) speed,  (3)heading  =  DL  1  2  3  FLID  enter)? 

yes _  no _  can't  tell _ 


Comment? 


16.  How  best  to  design  menu  text? 

Separate  menus  (AA,  IA,  speeds,  headings,  free  test) 
One  menu  for  stringing  Xing,  IA,  speeds,  headings) 

A  free  text  menu  (of  typical  free  text  messages) 

Comment? 


17.  For  the  Data  Link  services  tested,  what  would  be  the  projected 
effect  of  Data  Link  at  your  position  on  the  following: 

(Check  box,  leave  blank  if  cannot  project.) 

PROJECTED  EFFECT  ON  VG  G  SG  F  SP  P  VP 

System  capacity  I  I  I  I  I  I  I 

_ I _ I _ I _ I _ I _ I _ I 

Keeping  the  picture  I  I  I  I  I  I  I 

_ I _ I _ I _ I _ I _ I _ I 

Weather  handling  I  I  I  I  I  I  I 

_ I _ I _ I _ I _ I _ I _ I 

R-D  controller  coordination  I  I  I  I  I  I  I 

_ I _ I _ I _ I _ I _ I _ I 

Sector  interphone  coordination  I  I  I  I  I  I  I 

_ I _ I _ I _ I _ I _ I _ I 

Pilot-controller  communications  I  I  I  I  I  I  I 

_ I _ I _ I _ I _ I _ I _ I 

A/C  conflicts  (system  errors)  I  I  I  I  I  I  I 

_ _ I _ I _ I _ I _ I _ I _ I 

Flight  strip  marking/handling  I  I  I  I  I  I  I 

_ _ I _ I _ I _ I _ I _ I _ I 

Overall  efficiency  (speed)  I  I  I  I  I  I  I 

_ _ I _ I _ I _ I _ I _ I _ I 

Overall  safety  I  I  I  I  I  I  I 

_ _ I _ I _ I _ I _ I _ I _ I 

Overall  controller  workload  I  I  I  I  I  I  I 

_ _ I  I  I  I  I  I  I 
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Comments? 


18.  During  a  handoff,  should  the  receiving  controller  also  see  the 
up  arrow  in  the  FDB  when  a  TOC  is  sent  to  the  pilot? 

yes _ ,  no _ ,  can't  determine _ , 


Comment? 


19.  When  things  get  busy  is  Data  Link  _  more  effective 

_  less  effective 

_  equally  effective 

can't  determine 


Comment? 


20.  Since  the  pilot  has  the  new  frequency  on  display  for 
reference,  does  he  need  to  voice  check  in  on  the  new  frequency? 
yes _ ,  no _ ,  can't  tell _ 


Comment? 


21.  Is  there  a  need  for  selectable  automatic  TOC  (i.e.,  change 
frequency  message  is  uplinked  when  receiving  controller  accepts 
handoff)  on  a  per  aircraft  basis?  Possible  entry  could  be  adding 
"S"  to  handoff  initiation,  yes _  no _  can't  tell _ 

Comment? _ _ _ 


22.  Using  Data  Link  should  leave  no  "gaps"  in  the  system.  Did  you 
see  any  places  where  gaps  in  positive  control  could  occur?  If  so, 
any  ideas  as  how  to  prevent  them? 
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23.  What  about  Data  Link  do  you  like  best,  and  why? 


24.  What  about  Data  Link  do  you  like  least,  and  why? 


25.  What  future  services  would  you  like  to  see  Data  Link  do? 
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GAT  SCENARIOS 


The  pilots  flew  two  different  scenarios  at  altitudes  and  speeds 
(maximum  of  180  knots  (kts)  representative  of  general 
aviation  aircraft  operating  under  Instrument  Flight  Rules  (IFR) 
conditions.  The  field  elevation  for  the  General  Aviation 
Trainer  (GAT)  simulator,  though  not  representative  of  the 
Atlantic  City  (ACY)  -  Philadelphia  area,  was  set  at  5,000  feet 
(ft)  mean  sea  level  (m.s.l.).  This  allowed  the  pilots  to  reach 
the  assigned  en  route  cruising  altitudes  (12,000  ft  for  scenario 
#1  and  11,000  ft  for  scenario  #2)  in  an  accelerated  fashion. 
Before  each  run  began,  the  subject  pilots  were  issued  a  clearance 
delivery  by  the  inflight  observer  and  given  instructions 
to  make  requests  for  weather  (WX) .  In  voice  runs,  pilots  tuned 
to  ACY  Flight  Service  Station  (FSS)  for  destination  terminal  WX 
and  en  route  winds  aloft.  In  data  link  runs,  pilots  were 
instructed  to  use  the  Teledyne  display.  Experimenters  provided 
ground  taxi  instructions  (to  the  departing  runway,)  tower, 
departure  and  approach  controls. 

SCENARIO  #1. 

GAT  pilots  flew  a  westbound  route  totaling  93  nautical  miles  (nmi) 
(see  figure  F-l) .  They  departed  at  an  initial  heading  of  270  and 
were  cleared  to  8,000  ft  m.s.l.  Before  reaching  the  assigned 
altitude,  departure  handed  off  N8417P  to  Universal  Data  Set  (UDS) 
Center.  Shortly  after  inception  into  the  simulation  (control  from 
UDS  Center)  ,  the  pilots  were  instructed  to  climb  to  the 
assigned  altitude  of  12,000  ft  (or  given  an  interim  altitude  of 
10,000  ft,  then  the  assigned  altitude  of  12,000  ft).  After 
level  off  and  before  reaching  the  sector  boundary,  the  inflight 
observer  prompted  the  pilots  to  make  a  request  for  any  pilot 
reports  (PIREPS)  in  the  area.  In  voice  runs,  since  the  GAT 
simulator  had  one  radio,  the  pilots  requested  from  UDS  Center  a 
change  of  frequency.  The  controller  issued  the  instructions  to 
"...report  back  up  in  3  minutes."  Similarly,  after  hand-off 
to  the  next  controlling  sector,  the  inflight  observer  prompted 
the  pilots  to  make  a  request  for  a  surface  observation  at  the 
destination  airport.  On  the  final  approach,  UDS  Center  instructed 
the  pilots  to  descend  to  7000  ft  m.s.l.  (2000  ft  above  ground  level 
(AGL)  for  a  straight-in  Instrument  Landing  System  (ILS)  approach 
to  the  destination  airport.  The  controller  issued  a  final  command 
to  contact  approach  control  within  15-20  nmi  from  the  airport. 
A  total  of  four  heading  changes  were  made  during  this  scenario. 
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SECTOR  03  (FREQ  133.0)  I  LOW  SECTOR  17  (FREQ  133.2) 


T  PLAN  SCENARIO 


SCENARIO  #2. 


The  scenario  was  run  in  an  easterly  direction  with  one  less 
heading  change  and  an  additional  5  nmi  of  flight  (see  figure 
F-2 )  .  The  inflight  observer,  as  in  scenario  #1, 
instructed  the  pilots  to  make  two  WX  requests  before  release  and 
two  additional  requests  while  inflight.  The  initial  clearance 
altitude  and  heading  were  9,000  ft  and  090,  respectively.  Upon 
reaching  9,000  ft,  the  simulator  appeared  in  the  simulation. 
Air  Traffic  Control  (ATC)  instructions  included  two  altitude 
assignments  from  9,000  to  11,000  ft  and  from  11,000  to  7,000 
ft.  A  frequency  change  (sector  handoff)  was  issued  midway 
in  the  flight.  Total  simulation  time  for  each  run  was 
approximately  40  to  45  minutes  With  10  minutes  accounting 
for  preflight  check,  clearance  delivery,  and  ground/tower/ 
departure  procedures  as  described  earlier. 

B-727  SCENARIOS 

The  B-727  flight  plan  followed  an  eastward  route  from 
Philadelphia  to  Atlantic  City  (see  figure  F-3) .  Along  with 
EAL55,  a  number  of  other  simulated  aircraft  flew  the  same  route; 
8  aircraft  in  scenario  #1  and  11  aircraft  in  scenario  #2. 
Special  overflight  routes  were  incorporated  in  the  scenarios  to 
produce  potential  conflicts.  It  was  the  nature  of  these 
conflicts  and  the  presence  of  additional  high  altitude  aircraft 
which  caused  variations  in  the  amount  and  type  of  ATC 
instructions  given  to  EAL55  in  any  one  run.  (The  GAT  received 
consistent  amount  of  instructions  across  all  runs.) 
Additionally,  the  number  of  individually  controlled  sectors  along 
the  route  for  EAL55  decreased  from  3  to  l  for  scenario  #1  and 
scenario  #2,  respectively.  This  was  due  to  the  separation  of  the 
UDS  Center  airspace  into  two  distinct  environments;  one 
representing  a  simulated  departure  airspace  and  the  other  a 
simulated  approach  airspace. 

SCENARIO  #1. 

EAL55  was  released  into  the  simulation  at  12,000  ft  with  an 
initial  heading  of  038.  The  initial  airspeed  and  climb  rate 
were  325  kts  and  800  ft/min,  respectively.  The  relatively 
low  rate  of  climb  was  used  so  as  to  cause  potential  conflicts 
with  the  overflight  routes  (to  create  workload  for  the 
controllers) .  The  scenario  was  198  nmi  in  length  and  required 
nine  heading  changes.  EAL55  started  out  in  a  low  sector  with 
instructions  to  climb  to  23,000  ft.  A  frequency  change  from  a 
low  to  high  sector  usually  proceeded  an  assignment  to  25,000  ft. 
The  high  sector  controller,  in  turn,  instructed  EAL55  to  fly 
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VOR  114.2 

Initial  Point  PHL 
12,000  feet 
325  IAS 

Scenario  1  HOG  038 
Scenario  2  HDG  068 


FIGURE  F-3 .  B-727  SCENARIOS 


to  27,000  ft.  After  level  off  at  the  cruising  altitude  (maintained 
no  longer  than  5  minutes  in  any  run) ,  EAL55  received  another 
frequency  change  to  a  high  sector.  The  controller  then  issued  an 
altitude  assignment  to  24,000  ft,  followed  by  a  frequency  change 
to  a  low  sector  controller  who,  in  turn,  issued  a  final  assignment 
of  12,000  ft. 

SCENARIO  #2. 


EAL55  was  released  at  12,000  ft  with  an  initial  climb  rate  of 
800  ft/min  and  an  airspeed  of  325  kts.  Scenario  #2  was  the  same 
as  scenario  #1  except  for  an  initial  heading  of  068.  Scenario 
#2  was  194  nmi  in  length  and  required  eight  heading  changes.  As 
noted  by  the  initial  low  altitude  (12,000  ft),  EAL55  was  part 
of  the  departure  airspace.  Because  of  the  new  sectorization, 
EAL55  was  exposed  to  only  one  handoff  (UDS  Center  low-to-high) 
which  occurred  midway  in  the  flight.  The  departing  sequence  of 
ATC  instructions  paralleled  those  of  scenario  #1.  After  reaching 
an  assigned  altitude  of  27,000  ft,  the  number  of  ATC  instructions 
decreased  and  the  type  of  instructions  varied  with  each 
controller  group.  Though  both  scenarios  represented  medium  to 
high  traffic  loads,  scenario  #2  represented  an  increased 
level  of  potential  ATC  conflicts. 
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To  counteract  the  difference  in  number  of  frequency  changes  in 
each  scenario,  a  simulated  frequency  change  was  issued  prior  to 
EAL55 1 s  release  into  scenario  #2.  This  was  accomplished  by 
having  the  inflight  observer  instruct  the  first  officer  to  touch 
"SELF  TEST"  on  the  Teledyne.  Touching  SELF  TEST  causes  a 
"RUNNING"  followed  by  "PASSED  !"  indication  (refer  to  appendix  G, 
figure  G-7) .  Upon  returning  to  MAIN  the  annunciation  "TOUCH  FOR 
ATC  MESSAGE"  was  displayed  which  contained  the  simulated 
frequency  change.  This  was  an  attempt  to  add  to  the  workload  to 
correct  for  the  decrease  (from  scenario  #1)  in  the  number  of 
controlled  sectors  in  the  flight  plan. 
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TELEDYNE  SPECIFICATIONS 


The  display  device  used  was  a  Teledyne  Interactive  Display  Unit 
(IDU) ,  Model  2229346023,  upon  which  had  been  programmed,  text 
appropriate  to  the  service  being  used.  See  figure  G-l. 

SPECIFICATIONS. 

Type:  Light  emitting  diodes  (LED),  32  LED's/inch,  128  by  118  LED's 

Interactive  Characteristics:  via  24  X  &  21  Y  axis  infrared  beams 

Output:  Standard  ASCII  character  set  Electrical  Interface  RS-232 

The  cockpit  of  the  GAT  and  B-727  (with  the  Teledyne  installed) 
can  be  seen  in  figures  G-2  and  G-3,  respectively. 

FORMAT  CHARACTERISTICS 

The  opening  or  main  menu  format  is  shown  in  figure  G-4 .  This 
menu  allowed  for  selecting  various  weather  products,  messages 

received,  and  user  system  set  up.  A  location  identifier  (LOCID) 

menu  (figure  G-5)  allows  the  user  to  select  a  WX  station  of 

interest.  In  this  design,  a  list  of  six  LOCID' s  was  presented;  an 
on-screen  keyboard  option  (figure  G-6)  was  also  available.  The 
system  set  up  menu  (figure  G-7)  allowed  various  system  parameters 
to  be  adjusted.  Finally,  on  the  main  menu,  a  message  list 
option  (figure  G-8)  allowed  retrieval/review  of  the  eight  most 
recent  WX  or  ATC  messages. 

The  package  of  services  for  this  evaluation  included  two  ATC  and 
four  weather  services.  An  example  of  each  service  format  is 

presented  below  along  with  a  brief  description.  The  example 
formats  are  representative  of  those  seen  by  the  GAT  pilots  during 
the  evaluation.  The  B-727  formats,  while  similar,  were  slightly 
different  to  reflect  the  altitude  environment  difference.  In 
addition  to  the  two  fixed  ATC  services,  the  subject  controllers 
were  able  to  create  and  send  ATC  oriented  "free  text"  (20- 
character  limit)  messages,  e.g.,  heading  or  speed  changes. 

WEATHER. 

Terminal  Forecast  (FT)  -  A  2 4 -hour  prognosis  of  weather 
conditions  within  the  immediate  vicinity  of  a  selected  location; 
includes  sky  and  ceiling,  visibility,  weather,  obstructions  to 
vision,  cloud  heights,  and  whether  visual  or  instrument 
conditions  are  expected  (see  figure  G-9) . 

Winds/Temperatures  Aloft  (FD)  -  A  report  of  projected  winds  and 
temperatures  for  a  range  of  altitudes;  includes  wind  speed,  wind 
direction,  and  temperature  (see  figure  G-10) . 
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DATALINK  MAIN  MENU 

—  TERMINAL  FCST 

—  SURFACE  OBS 

—  WIND/TEMP 

-*  REQUEST  PIREP 

—  MESSAGE  LIST 

—  USER  SET  UP 


FIGURE  G-A.  THE  MAIN  MENU 


LOCATION  LIST 


TERM  FCST  FOR:  ENO 


-*  ACY 

-*  DQO  —  ENO 

—  MIV 

—  RBV  —  RHM 

—  SEND 

-*  KEYBOARD 

or  MT  1 

--------  OCIX  1  ! . . 

MAIN 

FIGURE  G-5 .  THE  LOCATION  IDENTIFIER  (LOCID)  MENU 
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TERMINAL  FCST  :  AGC 
A  B  C  D  E  F  G 

H  I  J  K  L  M  N 

O  P  Q  R  S  T  U 

V  W  X  Y  Z 

CLR  —  BKS  —  SPC  —  SEND 
********  SENT  !  ******** 

MAIN 

FIGURE  F-6.  THE  KEYBOARD  MENU 

USER  SET  UP 

AUDIO  DEMO 
LOGO 

TONES  ON  -*  TONES  OFF 

SELF  TEST  RUNNING 

PASSED 

DARK  BRIGHT  MAIN 

FIGURE  G-7 .  THE  USER  SET  UP  MENU 
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RECEIVED  MESSAGES 
ALT  ASSIGN  1333  UNA 
WND/TMP  ACY  1343  ACK 
T  FCST  ACY  1437  ACK 
SURF  OB  ACY  1438  ACK 
SURF  OB  AGC  1445  ACK 
PIREP  ACY  1510  ACK 
PIREP  MIV  1512  ACK 
FREQ  CHANGE  1515  WIL 
MSG  LST  MAIN 


FIGURE  G-8 .  A  TYPICAL  MESSAGE  LIST  FORMAT 


TERMINAL  FCST  DQO 

20TH  1800Z  TO  1800Z 
1800Z-0100Z 

SKY:  20  OVC 
0100Z-1200Z 

SKY:  15  OVC 
1200Z-1800Z 
MVFR 


MAIN 


FIGURE  G-9 .  A  TYPICAL  TERMINAL  FORECAST  FORMAT 


Surface  Observation  (SA)  -  A  report  of  current  ground  weather  at 
a  selected  station;  includes  sky  conditions,  ceiling,  visibility, 
weather,  obstructions  to  vision,  wind  direction/speed,  altimeter 
setting,  and  additional  remarks  (see  figure  G-ll) . 

PIREPS  (UA)  -  Pilot  reports  of  inflight  conditions  which  may 
include  information  on  sky  cover,  flight  visibility,  flight 
weather,  and  indications  of  icing  or  turbulence,  etc.  (see  figure 
G-12 )  . 

AIR  TRAFFIC  CONTROL. 

Altitude  Assignment  (AA)  -  An  ATC  command  to  change  to  assigned 
altitude.  The  command  may  also  .contain  time  or  crossing 
references  and,  if  required,  an  altimeter  setting  (see  figure 
G-13 )  . 

Frequency  Change  (FC)  (or  Sector  Hand-off)  -  An  ATC  command  to  a 
pilot  to  change  to  a  new  traffic  control  agency/center  frequency 
(see  figure  G-14) . 


DATA  LINK  SWITCHING  LOGIC 


WEATHER. 

1.  Downlink  WX  Request.  On  the  MAIN  menu,  select  the  WX  product 
of  interest;  a  menu  of  LOCID'S  appears.  The  service  requested 
is  displayed  above  the  menu  of  available  locations.  After 
selecting  the  desired  location,  the  three  letter  identifier  is 
displayed  next  to  the  service  requested;  e.g.,  TERM  FCST  FOR: 
AGC  (Allegheny  County) .  Once  a  location  is  entered,  touch 
the  "SEND"  command.  A  confirmation  that  the  request  has  been 
sent  (*****  SENT!  *****)  appears  briefly  on  the  display. 

a.  Subsequent  selections  of  a  location  in  the  LOCID  menu 
clears  the  previous  location  entered. 

b.  If  desired,  an  alternative  mode  of  selection  exists 
through  the  "KEYBOARD"  command  on  the  LOCID  menu.  The  KEYBOARD 
option  was  not  used  in  the  evaluation. 

c.  Once  "SEND"  has  been  selected,  the  LOCID  menu  remains 
until  selecting  "MAIN." 

d.  After  a  request  and  upon  returning  to  MAIN,  a  "?"  and 
the  location  requested  will  appear  next  to  the  requested  service 
to  denote  a  pending  WX  message  (see  figure  G-15) . 
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WIND/ TEMP  ALOFT  RBV 
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FIGURE  G-10 .  A  TYPICAL  WINDS/TEMPERATURE  ALOFT  FORMAT 


SURFACE  OBS  DQO 

TIME:  1254Z 

SKY  :20  OVC  VIS  :5 

TEMP  :49  DEW  PT  :38 
WIND:210/5 
ALTIMETER:  30.19 


_ MAIN 

FIGURE  G-1I .  A  TYPICAL  SURFACE  OBSERVATION  FORMAT 
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PIREP  RBV 

TIME:  1335Z 
ACFT:  KC135  FL280 
LOC  :360  30NM  RHM 
WIND:255/37  TMP:-30 


_ MAIN 

FIGURE  G-12 .  A  TYPICAL  PILOT  REPORT  (PIREP)  FORMAT 


ALTITUDE  ASSIGNMENT 

CLIMB  AND  MAINTAIN 

ALTITUDE:  16000 
ALTIMETER:  29.98 

WILCO  UNABLE 


FIGURE  G-13 .  AN  ALTITUDE  ASSIGNMENT  FORMAT 


FREQUENCY  CHANGE 
CONTACT  UDS  CENTER  ON 
NEW  FREQ:  122.50 
CURRENT  FREQ:  123.50 

WILCO  UNABLE 

FIGURE  G-14 .  A  FREQUENCY  CHANGE  FORMAT 


DATALINK  MAIN  MENU 
TERMINAL  FCST  ?  AGC 
SURFACE  OBS 
WIND/TEMP 
REQUEST  PIREP 

MESSAGE  LIST 
USER  SET  UP 


FIGURE  G-I5.  A  FORMAT  SHOWING  A  PENDING  WEATHER  MESSAGE 
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e.  If  a  request  for  WX  is  not  available,  "WX  NOT  AVAILABLE" 
is  briefly  displayed  near  the  bottom  and  the  system  returns  to 
MAIN  automatically. 


2.  Receive  Uplink  WX  Message.  A  three  beep  tone  and  the 
message  "TOUCH  FOR  WX  MESSAGE"  annunciate  the  arrival  of 
requested  WX  data  (see  figure  G-16) .  Upon  touching  "TOUCH  FOR 
WX  MESSAGE,"  the  WX  information  is  displayed. 

a.  Within  each  WX  message  (see  WX  formats)  the  word  "MAIN" 
is  displayed  at  the  bottom  right.  Touching  "MAIN"  will  toggle 
the  system  back  to  MAIN. 

b.  The  "?"  will  disappear  (in  MAIN)  next  to  the  WX  product 
after  viewing  the  WX  information.  The  LOCID  remains  until  a 
different  location  is  requested  for  that  service. 

c.  If,  while  waiting  for  a  WX  message,  an  ATC  message 
arrives,  the  prompt  "TOUCH  FOR  WX  MESSAGE"  will  not  appear  until 
after  responding  to  the  ATC  instruction.  If  an  ATC  message 
arrives  before  completing  a  request  for  WX,  the  request  will 
require  complete  reinitiation. 

AIR  TRAFFIC  CONTROL. 

1.  Receive  Uplinked  ATC  Instruction.  A  two  beep  tone 
(distinctive  from  WX  tone)  and  "TOUCH  FOR  ATC  MESSAGE"  (in 
reverse  video)  annunciate  the  arrival  of  ATC  instructions 
(see  figure  G-17) .  Upon  touching  "TOUCH  FOR  ATC  MESSAGE",  the 
ATC  message  is  displayed  along  with  the  options  "WILCO"  and 
"UNABLE . " 

a.  Upon  touching  "WILCO",  "*****  SENT!  *****"  is  displayed 
momentarily. 

b.  Upon  touching  "UNABLE",  "CONTACT  ATC  VIA  RADIO"  is 
displayed  momentarily,  followed  by  "*****  SENT!  ******  (see 
figure  G-18) . 

c.  After  "*****  SENT!  *****",  is  displayed  (as  in  la  and  lb), 
the  command  MAIN  replaces  the  WILCO  UNABLE  line.  The  ATC 
message  stays  on  the  screen  until  MAIN  is  selected. 

d.  If  more  than  one  ATC  message  is  queued,  subsequent  ATC 
messages  will  not  appear  until  the  current  ATC  message  has  been 
WILCOed  or  UNABLEd.  Every  ATC  message  in  the  queue  requires 
touching  the  TOUCH  FOR  ATC  MESSAGE  prompt. 
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DATALINK  MAIN  MENU 

—  TERMINAL  FCST  AGC 

— ►  SURFACE  OBS 

—  WIND/TEMP 

— *  REQUEST  PIREP 

—  MESSAGE  LIST 

—  USER  SET  UP 

TOUCH  FOR  WX  MESSAGE 

FIGURE  G-16 .  A  FORMAT  SHOWING  AN  AVAILABLE  WEATHER  MESSAGE 


DATALINK  MAIN  MENU 

—  TERMINAL  FCST  AGC 

—  SURFACE  OBS 

—  WIND/TEMP 

—  REQUEST  PIREP 

—  MESSAGE  LIST 

USER  SET  UP _ 

TOUCH  FOR  ATC  MESSAGE 

FIGURE  G-l 7 .  A  FORMAT  SHOWING  A  PENDING  ATC  MESSAGE 
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ALTITUDE  ASSIGNMENT 


CLIMB  AND  MAINTAIN 

ALTITUDE:  16000 
ALTIMETER:  29.98 
CONTACT  ATC  VIA  RADIO 


FIGURE  G-18 .  THE  DISPLAY  AFTER  AN  UNABLE  RESPONSE  TO  AN  ATC  MESSAGE 


MESSAGE  LIST. 

A  list  of  received  transactions,  both  ATC  and  WX,  can  be  viewed 
in  the  MESSAGE  LIST  upon  touching  the  "MESSAGE  LIST"  line  in  the 
MAIN  menu  (see  figure  G-8) .  The  "MAIN"  command  appears  on  the 
bottom  right  of  the  MESSAGE  LIST  menu. 

1.  Displayed  in  columnar  form  are  abbreviations  of  each  WX 
service  followed  by  the  LOCID,  time  received,  and  a  three-letter 
status  identifier  of  the  transaction.  An  ACK 
(Acknowledge)  or  NDA  (No  Data  Available)  distinguish  those  WX 
messages  that  were  available  (and  seen)  from  those  which  were  not 
available  for  the  station  requested. 

2.  An  abbreviation  of  ATC  messages  received  were  followed  by  the 
time  received  and  a  three-letter  description  of  the 
transaction:  WIL  (WILCO)  and  UNA  (UNABLE) . 

3.  If  desired,  the  complete  message  was  viewable  by  touching  the 
appropriate  line  in  the  list. 

4.  An  "MSG  LST"  command  appears  to  the  left  of  the  MAIN  command 
on  the  bottom  line  of  each  message  selected  through  the  MESSAGE 
LIST.  This  allows  one  to  return  to  the  MESSAGE  LIST  menu  to 
retrieve  another  message  or  return  to  MAIN. 
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5.  Eight  messages  can  be  displayed/retrieved  from  the 
MESSAGE  LIST.  Once  eight  message  transactions  have  been  made, 
each  succeeding  transaction  will  replace  the  existing  messages 
in  a  top-down  fashion. 

USER  SET  UP. 

Brightness  level  of  the  display  was  adjustable  through  the  USER 
SET  UP  menu  (retrieved  from  MAIN  menu) .  Other  options  on  the 
USER  SET  UP  menu,  Audio  Demo,  Logo,  and  Tones  on/off  were  available 
but  not  used  in  the  evaluation. 

GENERAL 

1.  The  system  did  not  permit  multiple  WX  requests.  A 
request  for  WX  required  completion  and  viewing  the  data  prior  to 
making  another  request. 

2.  When  TOUCH  FOR  WX  MESSAGE  or  TOUCH  FOR  ATC  MESSAGE  appeared, 
all  other  screen  functions  were  disabled. 

3.  If  a  TOUCH  FOR  WX  MESSAGE  is  displayed  and  an  ATC  message 
arrives,  the  message  TOUCH  FOR  ATC  MESSAGE  replaces  the  WX 
message.  After  completion  of  the  ATC  transaction,  the  WX 
message  reappeared. 

4.  With  all  touch  activity,  a  one-beep  tone  provided  audio 
feedback. 

5.  Where  shown  on  the  formats,  an  represented  a 

selectable  item.  The  bottom  line,  however,  did  not  have  an 
indication.  The  higher  level  commands  that  appeared  on  the 
bottom  line  of  the  display  were  enclosed  by  a  thin-lined 
rectangle,  allowing  the  area  to  be  more  discriminable. 
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GENERAL 


The  test  was  conducted  over  2  days.  Six  data  runs  were 
completed  each  day  at  each  location  for  a  total  of  24  runs.  Each 
pilot/crew  flew  three  runs;  two  with  Data  Link  and  one  using 
voice  (as  a  baseline  control  condition) .  The  ground  Data  Link 
group  also  allowed  time  for  up  to  four  supplemental  runs.  The 
airborne  group  at  NASA  participated  in  one  of  these  runs  because 
of  a  technical  problem  in  a  data  run.  The  airborne  group  at  the 
FAA  Technical  Center  participated  in  two  of  these  runs  for 
additional  data  (no  technical  problems  occurred) . 

An  attempt  was  made  to  exercise  each  of  the  four  weather  services 
during  each  run.  The  number  and  type  of  Air  Traffic  Control 
(ATC)  messages  were,  however,  variable.  Table  H-l  presents 
the  ATC  and  weather  services  exercised  in  each  run  for  GAT. 
Table  H-2  presents  this  information  for  the  B-727.  A  summary 
of  Data  Link  voice/weather  ATC  usage  for  the  GAT  and  B-727  is 
presented  at  the  bottom  of  table  H-2  (also  note  the  number  of 
voice  callbacks) . 


TABLE  H-l.  GAT  ATC/WEATHER  SERVICES  EXERCISE  BY  RUN 


Run 


Alt.  Assign 
Freq  Chg 
Free  Text 

T  Fcst 
S  Obs 
Wind/Temp 
PIREP 


12  3 


2  3  3 

111 
0  10 

111 

111 

111 

111 


4  5  6 

ATC 

12  2 
111 
0  0  2 

Weather 

111 

111 

111 

111 


7  8  9 


3  2  1 

111 
0  2  0 

111 

110 

111 

111 


11 _ 12 


2  13 

111 
2  2  1 

111 

111 

111 

111 
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TABLE  H-2.  B-727  ATC  WEATHER/ SERVICES  EXERCISED  BY  RUN 


8 _ 9 


Alt.  Assign 
Freq  Chg 


5  6 


4  3 


Free  Text 


1  3 


Weather 


T  Fcst  111! 

S  Obs  111: 

Wind/Temp  Oil: 

PIREP  0  0  0 

Totals  WX  Data  Link  57 

(Tables  1  Wx  Voice  22 

and  2)  -  callbacks  3 


ATC  Data  Link  106 
ATC  Voice  75 
-  callbacks  10 


OBJECTIVE 

The  objective  measures  of  performance  during  the  evaluation  were 
various  time  intervals.  Definitions  of  the  critical  time 
intervals  (in  both  Data  Link  and  voice)  follow.  Figure  H-l 
presents  the  time  interval  definitions  in  a  graphic  form  for 
clarity. 

TIME  INTERVAL  DEFINITIONS. 

Weather. 


MCT  -  Message  Creation  Time. 

Voice  -  The  interval  begins  with  the  pilot  activating  the  mike 
switch  (PTT-Push  To  Talk-ON) ,  during  which  a  request  for  weather 
is  made,  and  ends  when  the  pilot  releases  the  mike  switch  (PTT- 
OFF)  . 

Data  Link  -  The  interval  begins  when  pilot  touches  one  of  four  WX 
products  on  the  MAIN  menu;  included  is  the  time  required  in 
selecting  the  appropriate  LOCID.  The  interval  ends  upon  touching 
"SEND”  in  the  LOCID  menu. 
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FIGURE  H-1.  TIME  INTERVALS  DEFINED  (SHEET  1  OF  2) 


ST  -  System  Time. 


Voice  -  The  interval  begins  with  the  release  of  the  mj  ,ce  switch 
in  MCT  and  ends  with  the  FSS/Flight  Watch  release  of  the  mike 
switch  after  announcing  that  WX  is  available,  i.e.,  "I  have  your 
request,  over." 

Data  Link  -  The  interval  begins  with  touching  "SEND"  in  MCT  and 
ends  when  "Touch  for  WX  Message"  is  displayed. 

RL  -  Response  Latency. 

Voice  -  The  interval  begins  with  the  FSS/Flight  Watch  release  of 
mike  switch  and  ends  when  the  pilot  presses  mike  switch  to 
announce,  "Yes,  Flight  Watch,  go  ahead." 

Data  Link  -  The  interval  begins  with  the  annunciation  of  "Touch 
for  WX  Message"  and  ends  when  the  pilot  touches  the  prompt  "Touch 
for  WX  Message." 

MPT  -  Message  Processing  Time. 

Voice  -  The  interval  begins  when  FSS/Flight  Watch  activates  the 
mike  to  "read"  the  WX  message  to  the  pilot  and  ends  when  FSS 
releases  the  mike  after  reading  the  complete  message.  Any 
repeats  of  information  required  added  to  the  time  as  appropriate. 

Data  Link  -  The  interval  begins  with  the  appearance  of  requested 
weather  on  the  screen  and  ends  when  the  pilot  touches  MAIN  menu. 

Air  Traffic  Control 

RL  -  Response  Latency. 

Voice  -  No  counterpart  to  Data  Link. 

Data  Link  -  The  interval  begins  when  "Touch  for  ATC  Message" 
appears  on  the  display  and  ends  when  the  pilot  touches  the  prompt 
"Touch  for  ATC  Message." 

WUT  -  WILCO/UNABLE  Response  Time. 

Voice  -  The  interval  begins  with  controller's  PTT-ON  (during 
which  the  ATC  instruction  is  delivered)  and  ends  with  the  pilot's 
release  of  the  mike  switch  after  acknowledging  the  instruction. 
If  the  pilot  requested  a  "SAY  AGAIN,"  this  time  was  included. 

Data  Link  -  The  interval  begins  with  the  appearance  of  the  ATC 
Instruction  on  the  display  screen  and  ends  with  the  touch  of 
WILCO  or  UNABLE. 
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The  following  provides  an  example  of  the  voice  transactions  made 
between  the  pilot  and  FSS/Flight  Watch  as  implemented  in  the  test 
during  voice  runs. 

Action  Pilot: 

PTT-ON  (Start  of  MCT)  "Atlantic  City  Weather,  17P 

requesting  Dupont  (DQO)  Terminal 
Weather. " 

PTT-OFF  (End  of  MCT) 

(Start  of  ST) 

Weather: 

PTT-ON  "17  Papa,  please  stand  by." 

PTT-OFF 


PTT-ON  "17  Papa,  this  is  Atlantic  City 

Weather  with  your  request,  over." 

PTT-OFF  (End  of  ST) 

(Start  of  RL) 

Pilot: 

PTT-ON  (End  of  RL)  "Atlantic  City  Weather,  go  ahead." 

PTT-OFF 

Weather: 

PTT-ON  (Start  MPT)  "17P,  Dupont  Terminal  Weather  for  18Z  to 

01Z,  sky  coverage  at  2,000  feet,  over 
cast.  For  01Z  to  12Z,  sky  coverage  at 
1500  feet,  overcast.  For  12Z  to 
marginal  VFR  (Visual  Flight  Rules)  is 
expected.  Current  Atlantic  City 
altimeter  29.93." 

PTT-OFF  (End  of  MPT) 

Pilot: 

PTT-ON  "Atlantic  City  Weather,  17  Papa,  I  copy 

Dupont  Terminal  Weather."  -  Pilot  may 
read  back  weather  for  confirmation  at 
this  point. 

PTT-OFF 
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Figure  H-2  presents  the  pilot  time  response  data  to  ATC,  i.e., 
WILCO  or  UNABLE  time  differences  between  Data  Link  and  voice 
by  GAT  and  B-727.  Figure  H-3  presents  this  information  broken 
down  by  each  of  the  possible  ATC  services.  Figure  H-4 
presents  total  pilot  interaction  time  data  with  weather  services 
(message  creation  time  +  response  latency  +  message  processing 
time)  .  The  Overall  weather  -  Pilot  Response  time  data  by  Data 
Link  and  voice  (by  GAT  and  B-727)  are  presented  in  figure  H-5. 
Figure  H-6  presents  the  GAT  individual  weather  services  data. 
Figure  H-7  presents  the  B-727  individual  weather  services  data. 
The  overall  and  individual  weather  services  time  data  (Average 
and  Standard  deviation)  for  WX/ATC  for  GAT  and  B-727  are 
presented  in  tables  H-3  and  H-4,  respectfully. 

SUBJECTIVE 

The  subjective  data  will  be  presented  in  this*  section.  The 
questionnaire  was  administered  to  four  general  aviation  pilots 
and  four  airline  crews.  The  numerical  average  (AVG)  and  standard 
deviation  (STD)  was  calculated  for  each  rating  type  answer.  The 
AVG  and  STD  was  calculated  for  the  GAT  group,  the  B-727  group, 
and  the  combination  of  groups.  The  standard  rating  scale  used  in 
many  of  the  questions  is  presented  in  table  H-5. 

The  questions,  ratings  summary,  and  pilot  comments  are 
presented  below.  Each  question  is  preceded  by  a  brief 
observations  section.  Unless  otherwise  noted,  the  statistics 
were  conducted  on  the  combined  group  of  12  pilots.  Limited 
sample  size  prevented  any  tests  to  compare  the  general  aviation 
and  airline  pilot  responses. 

A  Chi-square  one  sample  test  (Statistical  Analysis  in  Psychology 
and  Education,  2nd  Edition,  1966  G. A.  Ferguson,  McGraw-Hill)  was 
conducted  on  the  data  from  questions  4,  25-28,  and  30.  The  null 
hypothesis  for  all  questions  was:  There  is  no  difference  in  the 
expected  number  of  responses  in  each  category,  and  any 
observed  differences  are  merely  chance  variations  to  be 
expected  in  a  random  sample  from  the  rectangular 
distribution  where  all  frequencies  are  equal.  All  p  values 
were  at  or  below  the  0.05  level  of  significance.  This  would 
indicate  the  presence  of  a  preferred  response  (Yes-No) 
within  the  choices.  The  test  statistics  are  presented  at  the 
end  of  each  question. 

A  Kolroorgorov-Smirnov  one  sample  test  (Ferguson)  was  conducted  on 
the  pooled  data  from  questions  1,  9,  12,  13,  17,  and  22.  The 
null  hypothesis  (for  these  questions)  is  that  there  is  no 
difference  in  the  expected  number  of  responses  to  each  rating 
number,  and  any  observed  differences  are  merely  chance  variations 
to  be  expected  in  a  random  sample  from  the  rectangular 
distribution  where  all  frequencies  are  equal.  The  use  of  this 
test  reflects  the  nature  of  the  data;  i.e.,  rating  type  data. 
The  Chi-square  test  was  used  with  yes-no  type  data.  Table  H-6 
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DATALINK  VS.  VOICE 
Overall  -  ATC 


FIGURE  H-2.  PILOT  TIME  RESPONSE  TO  ATC 


DATALINK  VS.  VOICE 
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FIGURE  H-3.  ATC  SERVICES  -  PILOT  TIME  RESPONSE 


Data  Link  vs.  Voice 

TOTAL  PILOT  INTERACTION  TIME 
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FIGURE  H-4.  PILOT  INTERACTION  TIME 


DATALINK  VS.  VOICE 
Overall  -  Weather 


H-ll 


FIGURE  H-5.  WEATHER  -  PILOT  TIME  DATA 


GAT  DATALINK  VS.  VOICE 
TERMINAL  FORECAST  /  SURFACE  OBSERVATIONS 
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FIGURE  H-6.  GAT  WEATHER  RESPONSE  DATA  (SHEET  1  OF  2) 


GAT  DATALINK  VS.  VOICE 
WIND/TEMP  /  PIREPS 
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FIGURE  H-6.  GAT  WEATHER  RESPONSE  DATA  (SHEET  2  OF  2) 


B-727  DATALINK  VS.  VOICE 
TERMINAL  FORECAST  /  SURFACE  OBSERVATIONS 
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FIGURE  H-7.  B-727  WEATHER  RESPONSE  DATA  (SHEET  1  OF  2) 


B-727  DATALINK  VS.  VOICE 
WIND/TEMP  /  PIREPS 
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TABLE  H-3 .  GAT  TIME  DATA  (WX  AND  ATC) 

Time  in  Seconds 
Data  Link  -  Weather 

MCT  ST  RL  MPT 


Ava 

Std 

Avq 

Std 

Avq  Std 

Avq 

Std 

FT 

8.2 

4.9 

15.7 

1.7 

3.1  0.5 

10.9 

4.1 

FD 

8.3 

4.6 

13.6 

0.9 

3.4  1.4 

8.5 

2.6 

UA 

7.6 

4.5 

14.1 

1.6 

3.9  1.2 

10.3 

4.1 

SA 

6.1 

2.4 

14.5 

1.9 

3.9  2.2 

11.5 

3.4 

Overall 

7.6 

9.1 

14.5 

1.7 

3.6  1.4 

10.3 

3.6 

Voice  - 

Weather 

MCT 

ST 

RL 

MPT 

Avq 

Std 

Avq 

Std 

Avq  Std 

Avq 

Std 

FT 

9.4 

2.5 

21.1 

0.2 

2.8  0.4 

22.4 

1.4 

FD 

7.9 

2.1 

20.3 

1.5 

2.3  0.2 

13.1 

0.8 

UA 

6.9 

2.7 

21.2 

0.6 

3.6  2.0 

17.6 

1.5 

SA 

5.0 

1.0 

18.9 

1.7 

2.9  0.3 

18.4 

1.5 

Overall 

7.5 

2.6 

20.4 

1.4 

2.9  1.1 

17.8 

3.7 

Data  Link  -  ATC 

RL 

WUT 

Avq  Std 

Avq  Std 

AA 

3.7  0.7 

4.9  1.2 

FC 

3.6  0.4 

6.9  1.3 

Free  Text 

3.9  0.7 

6.9  2.4 

Overall 

3.7  0.6 

5.8  1.8 

Voice  -  ATC 

RL 

WUT 

Avq  Std 

Avq  Std 

AA 

8.7  2.1 

FC 

n/a 

10.8  3.0 

Free  Text 

11.3  4.5 

Overall 

9.9  3.1 
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TABLE  H-4.  B-727  TIME  DATA  (WX  AND  ATC) 


Time  in  Seconds 


MCT 


FT 

Avq 

5.5 

Std 

2.0 

FD 

8.3 

4.6 

UA 

8.0 

5.7 

SA 

7.1 

4.6 

Overall 

7.1 

4.1 

MCT 


FT 

Avq 

10.9 

Std 

FD 

13.5 

— 

UA 

— 

— 

SA 

8.8 

4.4 

Overall 

9.9 

3.9 

AA 

FC 

Free  Text 
Overall 


AA 

FC 

Free  Text 
Overall 


Data  Link  -  Weather 


ST 


Avg 

13.2 

Std 

2.2 

Avg 

4.2 

16.0 

5.3 

3.2 

16.1 

1.1 

3.5 

16.6 

6.2 

3.8 

15.9 

4.8 

3.7 

Voice 

-  Weather 

ST 

Avg 

5.0 

Std 

Avq 

2.0 

10.0 

— 

1.9 

8.9 

4 . 1 

2.0 

• 

00 

3.6 

2.0 

Data 

Link  - 

ATC 

RL 

Avq  Std 
3.9  1.5 

4.1  2.2 

4.2  2.8 

4.4  3.8 

Voice  -  ATC 
RL 

Avq  Std 
n/a 


RL 

MPT 

Std 

Avq 

Std 

0.7 

16.9 

10.3 

0.6 

15.0 

6.9 

0.6 

15.3 

5.7 

0.8 

13.7 

4.8 

0.7 

15.0 

6.6 

RL 

Std 

MPT 

Avq 

Std 

— 

21.2 

— 

— 

22.0 

— 

— 

— 

— 

0.8 

14.8 

10.1 

0.6 

17.1 

8.6 

WUT 

Avq  Std 

6.6  2.8 

9.5  3.2 

8.5  3.6 

7.9  3.3 


WUT 

Avq  Std 
9.9  5.7 

10.3  4.5 

9.0  4.3 

9.7  5.0 
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TABLE  H-5 .  STANDARD  RATING  SCALE 


Str.idard  Settle 

7  -  VERY  GOOD,  no  changes  are  necessary 
6  -  GOOD,  a  few  minor  changes  are  necessary 
5  -  FAIRLY  GOOD,  a  number  of  minor  changes  are  necessary 
4  -  FAIR,  both  minor  and  major  changes  are  necessary 
3  -  FAIRLY  POOR,  a  few  major  changes  are  necessary 
2  -  POOR,  a  number  of  major  changes  are  necessary 
1  -  VERY  POOR,  a  complete  redesign  is  necessary 


TABLE  H-6.  KOLMORGOROV-SMIRNOV  TEST  RESULTS 


Question 

#1 

Overall  Concents 

P_ 

Level 

Return  to  MAIN  Concept 

.  10 

P 

<.15 

Question 

#12 

Surface  Observations 

Information  Retrieval  Time 

P 

>.20 

Information  Clarity 

.  05 

< 

P 

<.10 

Return  to  MAIN  Concept 

P 

>.20 

Terminal  Forecast 

Information  Retrieval  Time 

.  10 

< 

P 

<.15 

Return  to  MAIN  Concept 

P 

>.20 

Pilot  Reoort 

Amount  of  Information 

.05 

< 

P 

<.10 

Return  to  MAIN 

.05 

< 

P 

<.10 

Wind/Temn  Aloft 

Information  Retrieval  Time 

.05 

< 

P 

<.10 

Amount  of  Information 

.  15 

< 

P 

<.20 

Return  to  MAIN  Concept 

.  15 

< 

P 

<.20 

Question 

#13 

Select  LOCID's  Menu 

Menu  of  LOCID's  Concept 

P 

>.20 

Time  Required  to  Input/Enter 

.  10 

< 

P 

<.15 

LOCID  Menu  of  Location  Codes 

.  10 

< 

P 

<.15 

Question 

#20 

MAIN  Menu 

Clutter 

.05 

< 

P 

<.10 

Amount  of  Information 

.  10 

< 

P 

<.15 

Question 

#22 

LOCID  Menu  Lav-out 

Rating 

.  05 

< 

P 

<.10 
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presents  those  questions  (and  factors)  in  which  the 
observed  levels  of  significance  (p  values)  were  greater  than 
0.05.  All  other  factors  within  these  questions  had  observed 
levels  of  significance  less  than  0.05;  this  would  indicate  a 
preferred  rating  value.  A  further  discussion  is  presented 
in  the  Observations  Section  of  each  question. 

The  observed  level  of  significance  or  p  value  is  the  probability 
(assuming  the  null  were  true)  of  observing  a  value  of  the  test 
statistic  that  is  at  least  as  contradictory  to  the  null 
hypothesis,  and  supportive  of  alternative  hypothesis,  as  the  one 
computed  from  the  sample  data  (Statistics,  1982,  J.T.  McClave  and 
F.H.  Dietrich,  II  Dellen  Publishing  Company) .  Usually,  the  null 
hypothesis  will  be  rejected  if  the  observed  significance  level  is 
less  than  a  fixed  significance  level  chosen  by  the  reader.  A 
common  significance  level  in  the  literature  is  0.05. 

A  binomial  test  (McClave)  was  performed  on  question  21 
(workload,  five  point  scale)  ;  ratings  of  1,  2  and,  3  were 
combined,  as  were  ratings  of  4  and  5.  With  this  combination, 
the  null  hypothesis  was  than  no  difference  would  exist  between 
the  1-2-3  and  4-5  categories.  The  results  of  this  test  are 
presented  in  question  21  Observations. 

ATC  SERVICES. 

OBSERVATIONS . 

Altitude  Assignment  -  On  all  factors  listed,  the  average  rating 
was  GOOD  (6)  or  above.  The  B-727  pilots  rated  factors  slightly 
lower  with  more  variability  in  the  responses.  In  general,  the 
comments  stress  the  fact  that  on-screen  data  reduces  the  chance 
for  error. 

Frequency  Change  -  The  GAT  pilots,  in  general,  rated  the  factors 
higher  than  the  B-727  pilots.  The  ratings  were  GOOD  (6)  and 
above,  except  for  appearance  and  ambiguity  by  B-727  pilots.  This 
is  probably  related  to  their  comments  on  being  hard  to  read  the 
numbers  and  the  use  of  "Contact  UDS  ..."  (a  contact  was  not 
actually  required) . 

Overall  Concepts  -  The  two  factors  were  rated  GOOD  (>6)  by  both 
GAT  and  B-727  with  the  exception  of  "Return  to  MAIN"  concept  by 
B-727  pilots.  These  pilots  did  not  like  the  requirement  to 
Return  to  MAIN  after  a  Data  Link  message  was  received  and  read. 
Their  comments  provided  potential  alternatives  to  the  tested 
logic. 
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Statistical  Results  -  All  observed  p  values  were  at  or  below  the 
0.05  level  of  significance  except  the  "Return  to  MAIN"  concept 
which  was  between  0.10  and  0.15.  The  test  indicates  that  the 
pilots  had  a  preferred  response  on  each  factor  (except  "Return  to 
MAIN").  In  most  cases  this  rating  was  a  "6"  or  "GOOD."  A 
few  minor  changes  are  necessary. 


Question  1.  For  each  of  the  two  ATC  services,  rate  the 
following  factors  associated  with  the  format.  Use  the 
standard  scale  provided  in  making  responses  and  circle  the 
appropriate  number.  Please  comment  in  the  area  provided  (use 
the  back  of  this  page  if  necessary) . 


Altitude  Assignment 


GAT 

B- 

•727 

Overall 

Avq 

Std 

Ava 

Std 

Avq 

Std 

Appearance 

6.25 

0.96 

6.25 

1.17 

6.25 

1.06 

Clutter 

6.5 

1.0 

6.0 

1.2 

6.17 

1.11 

Amount  of  information 

7.0 

0.0 

6.38 

1.16 

6.58 

0.9 

Information  clarity 

6.25 

0.5 

6.25 

1.39 

6.25 

1.14 

Ambiguity 

6.75 

0.5 

6.63 

1.06 

6.67 

0.89 

Time  to  interpret 

6.5 

1.0 

6.25 

1.04 

6.63 

0.98 

Comments: 


GAT  Pilots 


#1  I  believe  the  first  three  lines  could 
requiring  less  reading;  for  instance: 

Altitude  Altitude 

|—>  L—> 

16.4  7.3 


be  compressed. 


#2  When  altimeter  setting  is  "missing,"  it  should  be  spelled 
out  instead  of  just  -M-. 

#4  Use  of  gold  lettering  instead  of  green,  not  as  hard  on  eyes. 


B-727  Pilots 


#1  Altitude  assignments  were  easy  to  read  and  understand. 

#2  Given  by  phone  to  A1  Rehmann  at  Tech  Center. 

#4  Altitude  Assignment  very  good — nice  to  see  it  in  print — no 
chances  for  mistake.  In  contrast,  on  last  run,  no  Data  Link,  one 
altitude  assignment  was  misread! 
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#6  For  an  important  item  of  information  such  as  altitude 
change,  I  think  the  less  data  on  the  screen  the  better, 
i.e.,  only  new  altitude  on  the  screen  or  only  new  altitude  and 
climb  or  descend 

#8  Much  clearer  than  voice. 


Frequency  Chanqe/Sector  Hand-Off 


GAT 

B- 

•727 

Overall 

Avq 

Std 

Avq 

Std 

Avq 

Std 

Appearance 

6.25 

0.96 

5.88 

1.13 

6.0 

1.04 

Clutter 

6.25 

0.96 

6.0 

1.07 

6.08 

1.0 

Amount  of  information 

7.0 

0.0 

6.13 

0.99 

6.42 

0.90 

Information  clarity 

6.5 

0.58 

6.5 

1.07 

6.5 

0.90 

Ambiguity 

7.0 

0.0 

5.5 

1.69 

6.0 

1.54 

Time  to  interpret 

6.25 

0.96 

6.0 

1.6 

6.08 

1.38 

Comments : 


GAT  Pilots: 

#1  Once  again,  too  many  words  to  read. 

#2  Having  current  plus  new  frequency  can  eliminate  errors  by 
pilot  misunderstanding  new  frequency  or  forgetting  old 
frequency. 


#4  Same  as  above  (AA  comment) . 

B-727  Pilots 

#1  Clear  and  easy  to  understand.  Especially  like  the  recall 
feature. 

#4  The  numbers  are  just  a  bit  hard  to  read.  Had  to  look  twice 
to  make  sure. 

#7  Do  not  use  the  word  "contact"  as  this  implies  a  radio  call 
to  center  is  required.  Maybe  use  words  like  "switch  to"  or 
"change  to"  when  commanding  a  frequency  change. 

#8  Much  clearer  than  voice. 
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WILCO/UNABLE  concept 
Return  to  Main  concept 

Comments : 


Overall  Concepts 


GAT 

B-727 

Overall 

Mg 

Std 

Ava 

Std 

Avg 

Std 

6.25 

0.5 

6.29 

1.5 

6.27 

1.19 

6.25 

0.5 

5.14 

1.77 

5.55 

1.51 

GAT  Pilots 
None 

B-727  Pilots 

#1  I  saw  no  specific  reason  to  return  to  MAIN.  If  this  is 

necessary,  perhaps  it  could  "revert"  to  Main  after  delay  time. 

#3  Three  attempts  were  made  to  secure  WX  information  for  ACY 
with  no  response. 

#4  Not  sure  that  I  want  to  return  to  Main,  except  for  this 
exercise. 

#7  I  recommend  leaving  last  altitude  assignment  showing  on 
screen.  It  is  providing  useful  information.  Looking  at  MAIN 
menu  has  no  benefit. 

#8  Should  return  to  "whatever"  after  WILCO? 


AUDITORY  ANNUNCIATION. 

OBSERVATIONS .  In  general,  all  pilots  were  satisfied  with 
the  auditory  annunciation  of  incoming  ATC  Data  Link 
messages.  Two  B-727  pilots  provided  a  MAYBE  response  and 
expressed  concerns  with  (a)  high  workload  situations,  and  (b) 
the  startling  effect  the  tone  had.  Other  comments  were  made 
concerning  the  tone  in  a  real  cockpit,  missing  the  tones,  and 
number  of  tones  in  the  present  cockpit. 


Question  2 .  The  ATC  services  in  this  evaluation  were 
ground  initiated.  Given  the  fact  that  other  tasks  were  required 
of  you  in  addition  to  using  the  Data  Link  system,  were  you 
satisfied  with  the  auditory  annunciation  of  an  incoming  ATC 
service?  If  not,  explain  your  dissatisfaction. 
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GAT 

Yes  1,2 ,3, 4 

NO 

Maybe 


B-727 

1/3/4, 5, 7, 8, 
2,6 


Comments: 

GAT  Pilots 

#1  I'm  a  little  concerned  that  they  might  not  be  heard  in  a 
real  cockpit. 

#2  What  happens  if  pilot  misses  first  beeps  because  of  other 

tasks  and  does  not  include  the  display  in  his  scan? 

B-727  Pilots 

#1  The  only  problems  I  encountered  had  to  do  with  volume  levels 
in  the  simulator  (727).  There  were  no  ATC  audio  problems. 

#2  When  workload  high,  difficult  for  pilot  flying  to  stay  in 

information  loop. 

#3  The  auditory  warning  was  adequate  with  the  cockpit  noise 

level . 

#4  OK  for  this  experiment,  it  was  a  good  ear-catcher.  But 

there  are  already  too  many  bells  and  whistles  in  the  cockpit. 
Now  comes  TCAS  (Traffic  Alert  and  Collision  Avoidance  System) 

chirp  and  DL  Ding  Dong.  Maybe  flashing  screen  or  light  will 
suffice. 

#6  It  was  a  bit  startling.  Perhaps  a  spectrum  of  three 

different  noises  for  different  messages  could  be  useful. 

VISUAL  ANNUNCIATION. 

OBSERVATIONS .  The  visual  annunciation  of  ATC  Data  Link  messages 
was  considered  acceptable.  Two  pilots  provided  a  MAYBE 
response.  One  suggested  flashing  the  entire  screen  and 
providing  Master  Caution — like  light.  The  other  pilot 

suggested  a  change  in  terminology.  Concerning  automatic 
appearance  of  ATC  messages,  the  pilots  were  negative.  Four 
pilots  provided  a  MAYBE  response,  but  did  not  provide 
sufficient  comments  to  make  a  judgment  here. 


Question  3.  An  incoming  ATC  message  was  visually 
annunciated  by  flashing  "TOUCH  FOR  ATC  MESSAGE"  at  the  bottom 
of  the  screen.  Was  this  operation  logic  acceptable? 
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GAT 

Yes  2,3,4 

No 

Maybe  1 


B-727 

2, 3, 4, 5, 6, 7, 8 

1 


Comments : 

GAT  Pilots 

#1  Probably  flashing  the  whole  screen  as  well  as  some  sort  of 
master  light  similar  to  a  master  caution  light  would  be  an 
appropriate  message. 

#2  Something  flashing  may  catch  pilot's  eye,  even  if  tone  is 
missed  Sound/WILCO/Maybe  Read. 

B-727  Pilots 

#1  While  this  is  OK,  I  don't  have  to  be  told  to  "touch"; 
perhaps  "message"  or  "clearance"  would  be  acceptable. 

#3  After  the  auditory  warning. 

#4  Accepted  as  good. 

Or,  for  example,  should  the  ATC  message  automatically  appear? 

GAT  B-727 

Yes 

No  1  1,3, 4, 5, 6, 7, 8 

Maybe  2,3,4  2 

Comments: 

GAT  Pilots 

#1  If  you  are  reading  a  message,  you  might  want  to  finish  it 
first. 

#2  If  ATC  message  automatically  appeared,  it  should  flash  until 
pilot  WILCO' s/UNABLE's.  However,  by  making  pilot  touch  screen 
for  message,  they  do  not  get  into  reflex  of  Bell 
sou  sd/ 

Wil ;ox/Maybe  -  Pead. 

#3  If  it  were  flashing  and  also  had  the  audible  indication. 
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B-727  Pilots 


#1  Automatic  delivery  would  be  too  easy  to  neglect  or  miss 
altogether.  A  message  alert  with  appropriate  crew 

acceptance  should  be  required. 

#3  Any  message  should  follow  some  sort  of  getting. 

#4  Maybe  I  am  just  looking  at  a  WX  page  and  want  to  copy  that 
first;  or  just  want  to  take  extra  time  before  I  am  ready — but 
would  like  to  examine  this  later. 

#5  Pilot  should  be  given  the  option. 

#6  There  might  be  something  on  the  screen  that  I  need  at  the 

time  that  it  (the  ATC  message)  would  come  on. 

#8  Some  pilot  action  should  be  necessary  to  further 

substantiate  the  message. 

TERMINOLOGY . 

OBSERVATIONS .  The  WILCO/UNABLE  choices  were  understood  by 
all  pilots.  The  terminology  was  acceptable  by  all  pilots 
except  two.  This  dissatisfaction  centered  on  the  use  of  the 
word  "Contact. .. "  (in  a  frequency  change)  when  a  contact  was 

not  required.  "WILCO"  means  "Roger"  according  to  one  pilot, 

but  he  indicated  that,  either  way,  there  was  no  problem. 
The  test  statistic  p  value  indicates  a  preferred  choice,  i.e., 
YES  for  both,  part  A  and  B. 


Question  4.  After  an  ATC  message  was  transmitted,  you  were 
required  to  make  either  a  WILCO  or  UNABLE  response.  Did  you: 

GAT 

Yes  No 


(a)  Fully  understand  the  choices  as  presented  1,2, 3, 4 

(b)  Find  the  terminology  acceptable  1,2, 3, 4 

If  not,  specify  a  or  b,  and  explain  your  dissatisfaction. 
None 
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B-727 

Yes  No 


(a)  Fully  understand  the  choices  as  presented  1,2, 3, 4, 

5, 6, 7, 8 

(b)  Find  the  terminology  acceptable  1,3, 4, 6, 7, 8  2,5 

If  not,  specify  a  or  b,  and  explain  your  dissatisfaction. 

#1  You  may  use  any  phraseology  you  wish,  but  "WILCO,"  as  I 
understand  it,  means  "Roger."  No  problem  here. 

#2  Call  ATC  was  not  displayed. 

#4  This  is  a  good  way  to  acknowledge  a  message  understood  and 
will  comply  or  UNABLE. 

#5  When  Center  told  us  to  chang _  a  frequency,  it  said  to 
contact,  which  to  us  means  to  talk  to.  It  should  say  monitor  or 
change  frequency. 

The  Chi-square  test  yielded  significance  levels  as  follows  for 
the  differences  between  Yes  and  No  answers. 

4a.  Chi-square  =12.0  p  <  .01 
4b.  Chi-square  =5.33  p  <  .01 

LOGIC  OPTIONS. 

OBSERVATIONS .  The  "SENT"  indication  after  a  WILCO  or  UNABLE 
was  considered  desirable  by  almost  all  of  the  pilots.  There 
were  three  MAYBE  responses.  One  pilot  stated  that  if  the 
logic  was  such  that  WILCO/UNABLE  did  not  disappear  until  the 
system  accepted  the  "SENT"  command,  the  display  of  "SENT" 
would  be  unnecessary.  In  general,  a  confirmation  that  the 
system  accepted  the  input  was  considered  necessary.  The 
tested  system  did  not  display  "SENT"  for  a  long  enough  period  of 
time. 


Question  5.  After  WILCO  or  UNABLE,  a  message  flashed  near  the 


bottom  of  the  screen  "SENT. 
Please  comment. 


GAT 

Yes 

1,3 

No 

Maybe 

2,4 

Is  this  a  desirable  option? 

B-727 

1,2, 3, 4, 5, 7, 8 

6 
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Comments : 


GAT  Pilots 

#1  Absolutely  necessary.  Just  like  ATC  wants  confirmation  of 
the  pilot  transmitting  the  message,  so  does  the  pilot  need  to 
know  it  was  sent.  The  "SENT”  message  stayed  on  the  screen 
for  too  short  a  time.  That  is  true  for  the  WX  portion 
too. 

#2  Not  really  necessary.  After  the  Message  WILCO/UNABLE  was 
sent,  those  two  words  disappeared  and  the  MAIN  option  appeared. 
The  length  of  time  "SENT"  appeared  on  the  screen  was  very 
short.  If  logic  was  written  to  prevent  clearing  WILCO/UNABLE 
until  message  sent,  then  this  flasher  would  not  be  needed. 

#3  Some  means  is  necessary  to  assure  the  pilot  that  his 
response  was  sent. 

#4  Something  definitely  has  to  appear  letting  the  pilot  know 
the  reply  was  transmitted. 

B-727  Pilots 

#1  It  indicates  to  the  user  that  his  action  has  been  sent. 

#2  Needed  as  confirmation 
#3  There  is  a  need  for  confirmation 

#4  Nice  to  know  acknowledgement  was  sent.  If  this  is  recorded 
on  each  flight,  it  might  be  used  to  prevent  any  controversy 
later  if  a  flight  conflict  results. 

#6  I  find  the  logic  of  a  second  step,  i.e.,  press  SENT,  to  be 
nonconsistent  with  the  typical  one-step  response. 

#7  However,  I  would  leave  the  word  "SENT"  on  the  screen  just  in 
case  something  else  happened  that  instant  and  you  couldn't 
remember  whether  or  not  you  sent  it. 

#8  It  really  is  nice  to  know. 

ATC  Message  Logic  Options 

Having  the  ATC  message  remain  on  screen  after  WILCO  or  UNABLE  was 
desired  by  all  pilots  but  one.  This  pilot's  comment  was,  "An 
unnecessary  motion."  It  is  assumed  the  comment  refers  to  the 
Return  to  MAIN  step.  This  option  was  described  as,  "very 
desirable,  reinforces  message,  ...can  be  looked  at  as  often  as 
necessary,  etc." 
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Question  6.  After  WILCO  or  UNABLE,  the  ATC  message  remained 
until  you  touched  MAIN.  Is  this  a  desirable  option?  Please 
comment . 

GAT  B-727 

Yes  1, 2,3,4  1,2, 3, 5, 6, 7, 8 

NO  4 

Maybe 

Comments : 

GAT  Pilots 

#1  You  might  be  ready  to  comply,  but  maybe  "data  dump"  the 
frequency  or  altitude. 

#2  Reinforces  message.  If  pilot  unsure  of  altitude  or  heading, 
allows  them  to  reverify. 

#3  Very  desirable.  In  voice  communication,  you  must  give  this 
operation  immediate  attention  so  as  not  to  forget  the 
message.  With  this  Data  Link  format,  the  machine  remembers  for 
you. 

#4  It  gives  the  pilot  the  option  when  to  clear  that  message. 
B-727  Pilots 

#1  The  message  can  be  looked  at  as  often  as  necessary.  This  is 
good,  especially  for  winds  aloft  and  other  WX  data. 

#2  Pilot  flying  might  not  get  to  see  it  otherwise  (especially 
in  high  workload) . 

#3  This  is  very  desirable  to  avoid  mistakes. 

#4  An  unnecessary  motion. 

#8  Why  is  it  necessary  to  go  to  MAIN  to  receive  a  new  message? 
I  like  to  see  my  current  clearance  in  front  of  me. 


REQUEST  CAPABILITY  OPTION. 

OBSERVATIONS .  Most  all  pilots  want  to  be  able  to  request  a 
different  altitude.  About  half  want  a  frequency  change  request 
option.  The  comments  provide  justifications  for  desiring  the 
request  option,  e.g.,  icing,  turbulence.  The  pilots  also  made 
suggestions  for  other  ATC  service  request  options  that  would  be 
desirable. 
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Question  7.  All  ATC  services  (in  this  evaluation)  were 
ground  initiated.  Do  you  feel  a  need  to  have  a  request 

capability  for  either  of  the  two  ATC  services? 


Request  capability  desired  for: 
Altitude 
Assignment 
GAT  2,3,4 

B-727  1,2, 3, 4, 5, 6, 7, 8 


Circle  your  choice (s) 
Frequency 
Change 
3,4 

1,4, 5, 8 


Comments : 


GAT  Pilots 

#1  If  it  is  for  confirmation  of  assignment,  no,  I  don't  need 
it  because  it  is  in  message  recall.  However,  if  I  was  given  a 
verbal  assignment  and  needed  to  recall  it,  as  long  as  the  ground 
facility  put  it  in  the  computer,  then  I  could  get  it  via  Data 
Link.  Nice  option. 

#2  May  want  to  request  different  altitude  for  winds, 
turbulence.  Marginal  call,  because  if  icing  or  turbulence  is 
too  high,  pilot  should  make  PIREP  and  REQ  Altitude  Change. 

#3  Pilot  should  be  able  to  request  altitude  change  and  what 
altitude  he  wants.  Same  with  frequency  changes,  particularly 
with  WX. 


#4  Yes,  if  a  system  could  be  designed  for  the  pilot  to  request 
these  items.  However,  it  may  be  quicker  just  to  use  the 
radio. 


B-727  Pilots 

#1  Not  really  necessary;  voice  communication  can  be  used  for 
some  things.  The  nice  aspect  of  Data  Link  is  its  ability  to 
cut  down  on,  or  eliminate,  the  oversaturation  of  verbiage. 

#2  Necessary  by  voice  or  Data  Link. 

#3  There  is  frequently  a  need  for  altitude  change  due  to 
turbulence  or  WX  conditions. 


#4  "Or  Just  About  Anything."  I  feel  defenseless  and  can't 
fight  back!  The  capability  for  two-way  communication  is  a 
must!  I  might  have  an  urgent  need  to  do  something  like  deviating 
thunderstorm,  getting  out  of  turbulence,  etc. 

#5  Also  ground  control  and  clearance. 

#6  Also  other  capabilities,  i.e.,  (1)  new  heading  for 
thunderstorm  avoidance,  (2)  new  routing,  (3)  new  airspeed  for 
turbulence. 
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#7  Would  be  nice  to  request  a  different  altitude  using  this 
system. 

#8  (He  adds  and  circles)  Speed,  Heading,  Anything. 

TIME  REQUIREMENTS. 

OBSERVATIONS .  For  both  altitude  assignments  and  frequency 
changes,  the  time  one  diverts  from  flying  to  operate  Data 
Link  was  considered  acceptable  (or  minimal) .  No  excessive 
responses  were  made;  one  "Depends"  response  was  made.  The 
"Depends"  response  was  qualified  by  the  comment  that  verbal 
commands  may  be  quicker  in  certain  situations,  e.g.,  instrument 
conditions  or  approach  to  a  busy  airport.  One  pilot  stated 
that  voice  communications  also  require  diversion  from  flying 
(to  the  same  extent  as  Data  Link)  and  that,  overall.  Data  Link 
is  superior  to  writing  clearances,  etc. 


Question  8.  The  Data  Link  system  as  implemented  will  require 
one  to  divert  time/attention  from  the  flying  task.  For  each 
service,  state  whether  the  time  required  was:  minimal, 
acceptable,  or  excessive  or  Depends  on  ...?  Circle  the 
appropriate  response  for  each  service.  Please  comment  on 
those  services  requiring  excessive  time  and/or,  if  necessary, 
discuss  the  depends  on  ... 


Altitude  Assignment  Minimal 
GAT 

B-727  1,2, 4, 8 


Acceptable 
2,3,4 
3, 5, 6, 7 


Excessive 


Depends 

1 


Frequency  Change 

GAT 

B-727 


Minimal  Acceptable  Excessive 
2  3,4 

8  1,2, 3, 4, 5, 6, 7 


Depends 

1 


Comments : 


GAT  Pilots 

#1  In  an  intense  environment  (i.e.,  Instrument  Meteorological 
Conditions  on  an  approach  to  a  busy  airport) ,  a  verbal  condition 
would  be  quicker  to  decipher.  However,  there  may  be  ways  to 
overcome  it.  A  shorter  command  as  presented  in  question  1,  and 
maybe  a  WILCO/UNABLE  button  on  the  yoke  or  throttle. 

#2  Single  pilot  IFR  frequency  change  probably  source  of  too 
much  traffic  because  pilot  not  anticipating  correct  frequency  or 
does  not  have  pen  in  hand  to  copy  new  frequency. 
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B-727  Pilots 


#1  One  must  realize  that  voice  communication  requires  diversion 
from  the  flying  task  to  practically  the  same  extent  as 
manipulation  of  Data  Link.  Overall,  Data  Link  is  superior  to 
jotting  notes  for  ATC  clearances  and  frequency  changes. 

#2  Altitude  assignment  is  critical  for  safety  and  legality.  If 
missed  by  pilot  flying,  it  could  cause  a  critical  situation. 
Somehow  the  communication  of  this  information  within  the 
operating  crew  should  be  accomplished  so  there  is  no  doubt  that 
information  is  understood  and  acted  upon. 

#3  The  location  of  the  display  must  be  such  that  response  can 
be  made  without  undue  effort  by  the  pilot,  particularly  for 
single  pilot  operator. 

#4  (for  AA)  Like  very  much. 

(for  FC)  With  exception  of  poor  readability  on  my  part, 
attention  requirements  were  minimal. 

#5  Anytime  you  have  excessive  workload,  the  flying  task 
suffers. 


#6  Could  be  excessive  if  one  crew  member  sees  the  message  then 
cancels  it  before  the  other  Crew  Member  see  it,  thus  prompting 
the  second  Crew  Member  to  ask  for  the  message  to  be 
recalled. 

#8  Not  once  did  anyone  ask  what  did  he  or  she  say.  It  is  very 
clear. 
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OVERALL  ATC  DESIGN 


OBSERVATIONS . 

The  ATC-Data  Link  overall  design  was  rated  FAIRLi  GOOD  to  GOOD, 
i.e.,  a  number  of  minor  changes  are  necessary.  The  comments  were 
very  positive,  e.g.,  "...better  than  I  thought  it  would  be  at 
this  stage  of  development,"  "like  the  Data  I.ink  just  the  way  it 
is."  Changes  suggested  dealt  with  terminology,  number  of 
services,  information  flow  rates,  etc.  The  observed  level  of 
significance  was  less  than  0.05  suggesting  a  preferred  rating, 
i.e.,  "5,"  or  "Fairly  Good,"  a  number  of  minor  changes  are 
necessary. 


Question  9.  Improper/incomplete  system  designs  can  result  in 
pilot  errors.  Overall,  how  would  you  rate  this  Data  Link  system 
ATC  services  design.  Use  the  standard  scale  provided.  If 
you  feel  design  problems  exist,  specify  the  areas  you  find 
unsatisfactory,  i.e.,  where  the  potential  for  errors  is  most 

prevalent.  Please  provide  your  suggestions  on  how  we  might 
redesign  the  system  to  overcome  these  problems. 

System  Design  -  ATC  Rating 

GAT  B-727  Overall 

Aver  Std  Avq  Std  Mcj  Std 

5.5  1.0  5.38  0.92  5.42  0.^0 

Problem  areas/suggestions: 

GAT  Pilots 

#1  The  concept  is  great.  The  evaluation  is  a  good  start. 
Addition  of  services  available  needs  to  be  evaluated,  and 
approach  and  departure  (i.e.,  high  intensity)  control 
evaluations  are  necessary. 

B-727  Pilots 

#1  The  pilot  flying  (PF)  does  not  "hear"  clearances  from  ATC 
directly.  The  pilot-not-f  lying  (PNF)  operating  the  Duta  Link 
terminal  must  relay  altitude,  heading,  and  routing  clearances  to 
the  PF.  In  a  relaxed  operating  environment,  this  isn't  a 
problem.  Maybe  it  is  or  is  not  a  problem  in  a  high  workload.  I 
would  need  more  experience  with  the  system  to  offer  a  valid 
opinion. 

#2  Phone  conversation  with  Al  Rehmann,  et  al.  on  4/20/89. 

#3  Messages  received  rate  a  7.  Information  requested  rate 
much  lower.  There  is  too  much  delay  after  requesting 
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information,  before  assurance  is  received  that  the  system  is  in 
process.  This  applies  primarily  to  Weather  Information 
requested. 

#4  Like  this  system  very  much.  I  like  to  see  the  recall  of 
messages  received  (and  WILCOed) . 

#5  I  really  like  the  Data  Link  just  the  way  it  is,  except  for 
the  frequency  change.  You  must  distinguish  between  monitor  and 
talk/contact . 

#6  The  following  areas  could  be  questioned:  The  lack  of  hi-tech 
features,  i.e.,  flashing  lights  (LED),  horns,  and  whistles. 
For  example,  say  the  ATC  message  comes  on  the  screen.  It 
could  flash  until  one  crew  member  acknowledged  it  and 
then  go  to  steady.  Then  when  the  second  crew  member  touched 
it  or  acknowledged  it,  the  light  or  MSG  could  go  out.  I  think 
a  real  limit  in  this  system  is  what  occurs  when  one  crew 
member  sees  the  MSG,  acknowledges  it,  and  then  erases  it 
before  the  second  crew  member  sees  it  or  absorbs  it. 

#7  When  listing  messages  received,  list  most  current  at  bottom. 
Do  not  list  them  as  now,  where  they  are  not  in  chronological 
order. 

#8  It  is  much  better  than  I  thought  it  would  be  at  this  stage 
of  development.  Pilot  should  be  able  to  initiate  calls. 
Tower  and  Ground  Control  must  be  able  to  use  this  or  similar 
system,  especially  high  density  airports,  e.g.,  ORD,  ATL,  SFO, 
LAX,  JFK,  etc. 

SUGGESTED  SERVICES. 

OBSERVATIONS .  The  pilots  made  numerous  suggestions  on  ATC 
services  to  be  part  of  future  Data  Link  designs.  A  few  of 
these  were  heading  information,  various  clearances,  and  "any 
clearance  that  ground,  tower,  departure,  approach,  or  Air  Route 
Traffic  Control  Center  (ARTCC)  controller  can  give  now  over 
voice  should  be  able  to  do  visually"  (Data  Link) . 


Question  10.  The  design  of  a  major  system  normally  goes 
through  numerous  iterations.  What  ATC  services  would  you 
consider  an  absolute  requirement  in  future  designs  and/or 
as  part  of  operational  systems? 

Suggested  Services: 

GAT  Pilots 

#1  WX  map  and/or  ground  radar.  Amended  clearances  in  a 
combined  map/alphanumeric  form.  Heading  information.  A  listing 
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in  column  form  of  alt,  headings,  and  frequencies  (Say,  maybe, 
the  previous  four  listed) .  Turbulence,  icing,  thunderstorm, 
SIGMETS  information. 

#2  Clearance  delivery  or  at  least  squawk,  departure, 

initial  altitude,  reading. 

#3  Clearance  display  for  referral. 

#4  Ground  speed  requests. 

B-727  Pilots 

#1  I  see  no  reason  why  all  services  cannot  be  included  in  the 
Data  Link  system. 

#2  WX  Warnings — Shear,  Clear  Air  Turbulence  (CAT) ,  Microburst, 
Thunderstorms,  major  wind  gusts  (changes,  —  Operation 
restrictions. 

#3  Altitude  change  requests. 

#4  Traffic  advisory  and  crew  acknowledgement.  Traffic  in  sight 
or  no  contact.  Other  ATC  Advisories — Change  in  A/S,  change  in 
climb  or  descent  rates.  Deviations  from  route  as  cleared. 

#5  Need  to  add  ground  and  ATC  clearance  control. 

#6  Alt  Assign,  Hdg  Assign,  Route  Assign. 

#7  (1)  Crossing  altitudes  at  specific  Distance  Measuring 

Equipment  (DME's),  (2)  Known  traffic  delays. 

#8  Crossing  altitude  restrictions,  holding  clearances.  Any 
clearance  that  ground,  tower,  departure,  approach  or  ARTCC  can 
give  now  over  voice  should  be  able  to  do  visually. 
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UNWANTED  SERIVCES . 


OBSERVATIONS .  The  number  of  services  that  should  not  be  part  of 
Data  Link  were  minimal;  examples  are  emergency  calls  and 
traffic  information.  One  pilot  wanted  emergency  information 
by  both  voice  and  Data  Link. 


Question  11.  What  ATC  services,  if  any,  should  not  be  part  of 
the  Data  Link  system? 

GAT  Pilots 

#1  Emergency  calls  unless  they  were  a  backup  to  voice.  Traffic 
calls.  You  don’t  want  your  head  inside  the  A/C  when  there  is 
traffic  close  by. 

B-727  Pilots 

#1  None  really. 

#3  Course  deviation  for  WX,  traffic  information. 

#5  In  an  emergency,  I  would  want  it  transmitted,  both  via  Data 
Link  and  verbal. 

#6  Other  traffic  location (s) 

#8  He  should  not  be  able  to  remotely  control  any  airplane 
through  the  Auto-pilot. 

If  there  are  any  other  comments  you  would 
wish  to  express  on  the  ATC  portion  of  the 
Data  Link  system,  use  this  page  and  the  back. 

GAT  Pilots 

#3  I  would  like  to  see  the  message  list  function  as  a  scroll, 
rather  than  excess  information  starting  at  top  again. 

#4  Possibly  to  let  you  know  if  you're  in  sequence  behind 

another  aircraft. 

B-727  Pilots 

#1  My  only  request  would  be  positive  verification  between 

flight  and  ATC  regarding  clearance  acceptance. 

#3  (1)  The  use  of  Data  Link  would  decrease  the  possibility  of 

misunderstanding  messages  between  pilots  and  controllers. 

(2)  For  routes  that  require  position  reports  (as  over 
ocean) ,  it  would  greatly  decrease  pilot  and  controller  workload. 
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(3)  It  makes  acquisition  of  WX  information  much  easier  than 
the  present  system. 

#4  I  would  be  the  last  one  to  2ose  the  "party  line."  I  like 
the  chatter,  the  quips,  ATC  admonitions,  and  mainly  "How  goes  it" 
ahead  of  us,  especially  during  the  descent  and  approach  phase. 
Turbulence  levels,  good  ride  levels,  windshear  on  final,  good 
area  of  passage  between  thunderstorms,  flights  that  missed  on 
approach  ahead  of  us — all  those  nice  things  I  am  UNABLE  to  lose 
thanks  to  Data  Link.  But  that's  probably  inevitable  and  the  way 
of  progress. 

#6  I  believe  the  ATC  portion  of  the  Data  Link  should  strive 
toward  becoming  a  true  two-way  communicating  system  for  exchange 
of  as  much  system  information  as  possible. 

#8  Let's  all  work  to  get  this  in  the  system  as  soon  as 
possible.  I  believe  it  is  much  safer  than  voice. 

WEATHER  SERVICES. 

OBSERVATIONS ♦  This  question  dealt  with  the  four  Weather  Service 
formats.  Eight  factors  are  listed  for  each  service.  In 
general,  all  average  ratings  were  FAIRLY  GOOD  or  above.  The 
standard  deviation  was  fairly  high,  indicating  a  wider  range  of 
ratings.  The  "Return  to  MAIN"  concept  was  rated  lowest  in 
most  cases.  Amount  of  information  on  surface  observations 
and  terminal  forecasts  was  rated  lower  by  GAT  pilots.  The 
B-727  pilots  rated  amount  of  information  (Winds/Temp)  low. 
Several  comments  were  made  by  the  pilots  concerning  general 
format,  content,  amount  of  information,  and  logic. 

STATISTICAL  RESULTS.  All  observed  p  values  were  at  or  below  the 
0.05  level  of  significance  (except  those  noted  in  table  H-6)  , 
thus,  suggesting  a  preferred  rating.  This  rating  was 
predominately  in  "5"  or  "FAIRLY  GOOD." 


Question  12.  Rate  the  format  of  each  of  the  four  weather 
services  on  the  following  factors.  Use  the  standard  scale  in 
making  responses  and  circle  the  appropriate  number.  Please 
comment  on  each  service  as  needed  on  the  bottom  of  the  next 
page  (use  the  back  of  this  page  if  necessary) . 
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Surface  Observations 


GAT 

B- 

727 

Overall 

Avq 

Std 

Avq 

Std 

Avq 

Std 

Appearance 

5.5 

1.0 

5.88 

0.99 

5.75 

0.9 

Information  retrieval  time 

4.75 

1.89 

5.13 

1.46 

5.0 

1.54 

Clutter 

5.0 

1.41 

5.5 

1.69 

5.33 

1.56 

Information  clarity 

5.0 

1.41 

5.75 

1.28 

5.5 

1.31 

Amount  of  information 

5.0 

2.0 

5.75 

0.89 

5.5 

1.31 

Ambiguity 

6.0 

0 .82 

5.5 

'  *» 

5.67 

1-07 

Time  or  valid  times  concept 

6.5 

0.58 

5.88 

0.64 

6.08 

0.67 

"Return  to  MAIN"  concept 

6.25 

0.96 

4.5 

2.14 

5.08 

1.98 

Terminal 

Forecast 

GAT 

B-727 

Overall 

Avq 

Std 

Avq 

Std 

Avq 

Std 

Appearance 

5.5 

1.0 

5.88 

0.99 

5.75 

0.97 

Information  retrieval  time 

5.0 

2.0 

5.25 

1.67 

5.17 

1.7 

Clutter 

5.25 

1.5 

5.5 

1.41 

5.42 

1.38 

Information  clarity 

5.5 

1.73 

5.75 

1.04 

5.67 

1.23 

Amount  of  information 

4.75 

2.06 

5.5 

0.93 

5.25 

1.36 

Ambiguity 

6.25 

0.96 

5.38 

1.3 

5.67 

1.23 

Time  or  valid  times  concept 

6.25 

0.5 

5.88 

0.64 

6.0 

0.60 

"Return  to  MAIN"  concept 

6.25 

0.96 

4.5 

2.14 

5.08 

1.98 

PIREP 

GAT 

B-727* 

Overall 

Avq 

Std 

Avq 

Std 

Avq 

Std 

Appearance 

6.25 

0.96 

6.5 

0.58 

6.38 

0.74 

Information  retrieval  time 

5.25 

2.22 

5.5 

2.38 

5.38 

2.13 

Clutter 

5.5 

1.73 

6.5 

0.58 

6.0 

1.31 

Information  clarity 

5.5 

1.73 

6.5 

0.58 

6.0 

1.31 

Amount  of  information 

5.25 

0.96 

5.75 

1.26 

5.5 

1.07 

Ambiguity 

6.25 

0.96 

5.75 

1.26 

6.0 

1.07 

Time  or  valid  times  concept 

6.75 

0.5 

6.0 

0.82 

6.38 

0.74 

"Return  to  MAIN"  concept 

6.25 

0.96 

5.25 

1.71 

5.75 

1.39 

*Based  on  four  responses 

(two  crews 

did  not 

view 

PIREP  due  to 

technical  problem) . 


H-  37 


Wind/Temp  Aloft 


GAT  B-727*  Overall 


Ava 


Appearance  6 . 5 

Information  retrieval  time  5.75 
Clutter  6 . 0 

Info  clarity  (DIR-VEL-TEMP)  6.25 
Amount  of  info  (3  Altitudes)  6.5 
Ambiguity  6.5 

Time  or  valid  times  concept  6.75 
'•Return  to  MAIN"  concept  6 . 5 


*Based  on  six  responses. 


Std 

Ava 

Std 

Ays 

Std 

1.0 

6.0 

1.10 

6.2 

1.03 

2.5 

5.33 

1.86 

5.5 

2.01 

2.0 

6.0 

1.10 

6.0 

1.41 

1.5 

5.0 

2.0 

5.5 

1.84 

1.0 

4.67 

1.63 

5.4 

1.65 

1.0 

5.67 

1.37 

6.0 

1.25 

0.5 

5.83 

0.75 

6.2 

0.79 

1.0 

4.67 

1.63 

5.67 

1.5 

Comments : 


Surface  Observations 


GAT  Pilots 


#1  I  believe  some  jets  figure  their  T/0  data  on  Press  Alt  or 
Density  Alt. 

#2  Dividing  sky  line  so  that  visibility  has  a  line  of  its  own 
would  make  information  a  little  easier/faster  to  interpret. 


B-727  Pilots 


#1  Standard  WX  Bureau  information. 


#4  For  destination,  I  would  like  to  see  more  information 
— probably  same  as  on  latest. 

#5  Good. 


#6  Format  of  the  WX  report  is  a  bit  different  from  that  which  I 
am  used  to  seeing. 

#7  When  requesting  weather,  after  pressing  desired  station,  why 
hit  "Send"?  Seems  like  an  extra  step. 

#8  Again,  why  use  the  lower  portion  to  signal  pilots?  Pilot 
parenthesis  the  "Return  to  MAIN"  Concept  and  writes  "Pilot 
Option."  Use  similar  format  that  we  see  from  National  Weather 
Services.  Made  up  acronyms;  I  HATE  them. 
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Terminal  Forecast 


GAT  Pilots 

#1  The  addition  of  forecast  winds,  outside  air  temp,  and  min 
altimeter  would  be  nice. 

#4  Should  also  include  projected  visibilities. 

B-727  Pilots 

#1  Standard  WX  Bureau  information. 

#4  OVC  5  miles  is  not  enough  information.  Need  to  know 
ceiling. 

#  5  Good . 

#6  Too  much  information  and  not  in  exactly  the  format  I  am  used 
to  seeing  in  the  various  formats  I  see  in  weather  offices.  You 
could  allow  the  forecast  for  all  the  time  periods  that  are  "post" 
(ago)  to  drop  off  the  screen  so  that  all  that  would  be  showing 
would  be  present  and  future. 

#7  Same  as  before. 

#8  Same  comments  as  before. 


PIREP 


GAT  Pilots 

#1  A  comments  section  that  might  include  turbulence,  icing , 

cloud  layers,  etc.,  would  be  helpful. 

#2  If  more  than  one  PIREP,  how  are  others  displayed?  Are  they 
queried,  or  is  only  the  most  recent  displayed? 

B-727  Pilots 

#1  UNABLE  to  answer  PIREP.  We  received  no  PIREPS  on  any  Data 
Link  run. 

#2  Got  none. 

#4  Good  useful  information  such  as  icing  levels,  turbulence, 

thunderstorm  activity 

#5  Good. 

#7  Same  as  before. 
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GAT  Pilots 
#1  Good. 


Wind/Temp  Aloft 


#2  Very  clear,  easy  to  interpret  display.  All  reports  should 
have  the  option  of  returning  to  the  request  stage  rather  than  all 
the  way  back  to  Main  Menu. 

#3  (General  Comment  for  all  WX  Products)  All  of  them  take  too 
long.  Also,  don't  need  touch  for  WX.  Just  display  the 
information  following  an  audible  alarm. 

#4  (FD's)  Good. 

B-727  Pilots 

#1  Would  like  altitudes  for  every  2,000  ft  from  18,000-41,000. 

#2  Here  is  an  opportunity  to  add  comments  based  on  computer 
modeling.  For  example,  if  there  is  a  large  wind  direction  or 
velocity  change  between  altitudes,  a  note  or  warning  could  be 
given  for  a  specific  area.  The  note  could  be,  e.g.  ,  expect  50% 
possibility  CAT  between  FL  250  and  FL  280  within  50  nmi  of  (Fix) 
or  (Coordinate) ,  expected  intensity — light. 

#3  (Remarks  apply  to  all)  The  presentation  requires 
considerable  effort  to  read  when  lengthy  messages  are  presented. 
The  letters/ numbers  produce  a  sort  of  halo  that  is  difficult  to 
see  through.  Short  messages  are  no  problem.  The  retrieval  time 
for  requested  messages  is  too  long  and  one  is  not  always  sure  the 
request  is  being  processed. 

#4  Good  information,  but  would  like  to  be  able  to  request  other 

levels  as  needed.  (Need  to  look  at  other  levels  than  the  three 

presented. ) 

#5  Need  more  altitudes.  Three  is.  good,  but  more  would  be 

better. 

#6  I  would  like  the  ability  to  request  Wind/Temp  aloft  "AT  F/L 
330  Only"  (for  example) . 

#7  Same  as  before.  Why  show  last  station  queried  on  MAIN 

menu?  It  tells  us  nothing  useful. 

#8  (Use  real  words,  no  acronyms  or  made  up  abbreviations.) 
This  applies  to  all  formats. 
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LOCID  MENU  DESIGN. 


OBSERVATIONS ♦  The  LOCID  menu  factors  were,  in  general,  rated 
higher  by  GAT  pilots  than  B-727  pilots.  The  menu  of  LOCID 
codes  was  rated  FAIR  by  both  groups.  The  comments  centered  on 
actual  implementation,  number  of  LOCID' s,  ambiguity  of  LOCID 
codes,  etc.  The  LOCID  menu  concept  and  time  to  input  factors 
were  rated  lower  by  B-727  pilots. 

STATISTICAL  RESULTS.  The  factor  "Menu  of  LOCIDS  concept"  had  an 
observed  p  value  greater  than  0.20.  This  indicates  a  variety  of 
ratings  from  the  pilots.  Similarly,  the  "Time  Required..."  and 
"LOCID  menu..."  faster  had  observed  p  value  between  0.10  and 
0.15.  All  other  factors  had  p  values  at  or  below  0.05  indicating 
a  preferred  response. 


Question  13.  The  weather  services  required  you  to  select  a 
location  identifier  (LOCID)  (e.g.,  LAX  or  SAT)  from  the  LOCID 
menu,  depress  enter  and  wait  in  the  LOCID  menu  while  the 
service  info  was  gathered.  The  LOCID  menu  displayed  the 
location  code  you  had  entered.  Rate  the  design  concepts  on  the 
following  factors.  Use  the  standard  scale  in  making  responses 
and  circle  the  appropriate  number.  Please  comment  in  the  area 
provided  (use  the  back  of  this  page  if  necessary) . 

GAT  B-727  Overall 


Avq 

Std 

Avq 

Std 

Avq 

Std 

Menu  of  LOCID 's  concept 

5.0 

1.41 

4.5 

1.85 

4.67 

1.67 

Time  required  to  input/enter 

5.5 

1.0 

4.86 

1.35 

5.09 

1.22 

Possibility  for  error 

5.75 

0.96 

5.57 

1.27 

5.64 

1.12 

Number  of  "keystrokes" 

5.0 

0.82 

5.71 

0.95 

5.45 

0.93 

LOCID  menu  of  location  codes 

4.75 

1.89 

4.5 

0.55 

4.6 

1.17 

LOCID  remaining  on  format 

6.0 

1.0 

5.0 

1.22 

5.38 

1.19 

(to  request  another  service 
at  the  same  location) 


Comments: 

GAT  Pilots 

#1  Concept  is  OK,  but  how  will  it  be  programmed?  The  time  to 
enter  is  no  problem.  Possibility  for  error  is  low,  at  least  with 
my  fingers.  Number  of  keystrokes  might  be  reduced  by  changing 
"S^nd"  to  "Correction." 

#2  This  is  an  area  where  both  concepts  could  be  used.  A  menu 
of  LOCID' s  for  the  local  area  and  the  keyboard  for  distant 
places. 
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#3  Could  not  possibly  list  all  potential  LOCID's.  Some  means 
is  necessary  for  pilot  to  enter  LOCID's. 

#4  There's  always  a  possibility  for  error  using  the  touch  type 
pad.  Turbulence  could  make  it  very  difficult  to  select  the 
correct  pad  the  first  time. 

B-727  Pilots 

#1  This  system  is  really  preLty  good.  Maybe  the  display  could 
be  cleaned  up  just  a  bit,  but  overall  it's  acceptable.  I  don't 
think  many  errors  would  come  from  this;  besides,  they're  easy  to 
catch  should  they  occur. 

#2  Since  there  is  no  airport  menu,  it  is  possible  not  to  know 
an  identifier  or  to  enter  the  wrong  identifier  (when  not  in  the 
on-screen  menu) .  This  could  lead  to  getting  weather  for  one 
location  which  you  believe  to  be  from  another. 

#3  The  LOCID's  concept  is  good.  However,  there  should  be 
capability  to  type  in  or  request  stations  that  may  not  be  listed. 
Time  required  to  obtain  information  seems  excessive.  The  number 
of  keystrokes  could  be  reduced. 

#4  Not  bad  system  and  fairly  easy  to  use,  but  I  believe  that  a 
quick/slow  scroll  through  all  identifiers  in  a  data  base  would 
be  less  prone  to  error  (such  system  exists  in  NAV  STAR-1, 
LORAN-C) . 

#5  The  only  problem  with  this  format  is,  let's  say  you  have  an 
emergency  and  have  to  divert  to  some  place  not  on  the  LOCl.i's 
menu.  It  would  be  nice  to  put  in  what  station  we  want  weather 
for. 

#6  Once  I  have  selected  the  location  from  the  LOCID  menu,  the 
request  should  be  sent  rather  than  having  me  then  press  the  sent 
button. 

#7  Seems  like  the  keyboard  concept  would  add  versatility  to 
system  without  cluttering  screen. 

#8  I  should  be  able  to  type  in  information  as  to  what  I  want. 
LOGIC  OPTIONS. 

OBSERVATIONS .  The  "SENT"  message  after  touching  SEND  was 
considered  a  very  desirable  option.  The  confirmation  it 
provides  that  the  system  is  working  is  critical.  The  length  of 
time  "SENT"  appears  was  too  short.  Using  the  word  "WAITING" 
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in  addition  to  "SENT"  was  suggested.  The  requirement  to 
touch  "SEND"  was  considered  an  unnecessary  extra  step  by  one 
pilot. 


Question  14.  After  selecting  a  location  identifier  and 
touching  "SEND,"  a  message  flashed  near  the  bottom  of  the  screen 
"SENT."  Is  this  a  desirable  option?  Please  comment. 

GAT  B-727 

Yes  1,2, 3, 4  1,2, 3, 4, 5, 7, 8 

No  6 

Maybe 

Comments : 

GAT  Pilots 

#1  However,  it  stayed  on  the  screen  for  too  short  of  a  time. 
Consequently,  I  went  back  to  the  MAIN  menu  to  look  for  the 
question  mark. 

#2  Since  the  system  waits  for  the  information  to  be  returned,  a 
"SENT,"  and  possibly  a  "WAITING"  message  are  helpful. 

#4  Again,  there  has  to  be  some  way  the  pilot  knows  his  request 
has  been  sent. 

B-727  Pilots 

#1  At  least  you  know  the  system  is  attempting  to  retrieve  your 
information. 

#2  Necessary  confirmation. 

#3  To  assure  message  went  out. 

#4  Definitely. 

#6  It's  an  extra  step  that  should  be  eliminated. 

#7  Do  not  need  "SEND"  button.  Pushing  Station  ID  should  send 
it  AUTO.  "SENT"  should  remain  on  screen. 

#8  Very  much. 

The  logic  in  which  weather  information  remains  on  screen  until 
pilot  input  was  considered  very  desirable.  The  ability  to  read 
the  message  more  than  once  and/or  copy  it  onto  paper  is 
necessary. 
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Question  15.  After  the  weather  information  is  displayed,  the 
message  remained  until  you  touched  main.  Is  this  a  desirable 
option?  Please  comment. 


GAT  B-727 

Yes  1 , 2 , 3 , 4  1,2, 3, 4, 5, 6, 7,8 

No 

Maybe 


Comments : 

GAT  Pilots 

#2  Option  should  be  added  to  go  back  and  reguest  some 
information  (product)  from  another  location. 

#3  Pilot  may  want  to  refer  to  it  between  other  chores. 

B-727  Pilots 

#1  Please  see  comments  in  question  1,  Overall  Concepts. 

#3  Any  message  should  remain  until  the  pilot  has  understood  and 
released  it. 

#4  Want  to  see  it  and  copy  it. 

#6  WX  has  to  be  read  more  than  once  to  assimilate. 

TIME  REQUIREMENTS. 

OBSERVATIONS .  For  all  weather  services,  the  time  diverted  from 
the  flying  task  was  considered  acceptable  by  most.  Cne  B-727 
pilot  indicated  both  minimal  and  acceptable.  One  B-727  pilot 
felt  all  services  required  excessive  time  because  of  historic 
data  that  provides  no  information. 


Question  16.  The  Data  Link  system  as  implemented  will  require 
one  to  divert  time/attention  from  the  flying  task.  For  each 
service,  state  whether  the  time  required  was:  minimal, 
acceptable,  or  excessive  or  depends  on  ...?  Circle  the 
appropriate  response  for  each  service.  Please  comment  on 
those  services  requiring  excessive  time  ana/or,  if  necessary, 
discuss  the  depends  on  ... 
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Surface  Observation 
GAT 

Minimal 

Acceptable 
1,2, 3, 4 

Excessive 

Depends 

B-727 

5,8 

1,2, 3, 4, 7, 8 

6 

Terminal  Forecast 

GAT 

Minimal 

Acceptable 
1,2, 3,4 

Excessive 

Depends 

B-727 

5,8 

1,2, 3, 4, 7, 8 

6 

Pilot  reports 

GAT 

Minimal 

Acceptable 
1,2, 3,4 

Excessive 

Depends 

B-727 

5,8 

1,2, 4,7, 8 

3,6 

Winds/Temo  aloft 

GAT 

Minimal 

2 

Acceptable 

1,3,4 

Excessive 

Depends 

B-727 

5,8 

1,3, 4, 7, 8 

2,6 

*B-727  #8  Chose  both  minimal  and  acceptable  for  all  services. 
Comments : 

GAT  Pilots 

#3  Although  acceptable,  an  improved  format  and  quicker  response 
would  greatly  improve  the  service. 

B-727  Pilots 

#1  When  one  turns  a  frequency  to  request  information  and  then 

listens  and  writes  that  information,  time  is  diverted  from  the 
flying  task.  The  Data  Link  method  of  information  retrieval  is 
superior  to  voice  and  pencil. 

#2  See  prior  FD  comments  (i.e.,  what  does  this  tell  me?). 

#3  The  information  is  adequate  again.  The  presentation  or 

matrix  produces  a  sort  of  halo  that  makes  messages  that  fill  the 
scope  difficult  to  read. 

#4  Time  used  was  no  more  than  it  would  take  for  a  verbal 

communication . 

#6  For  SA's,  trim  the  screen  information  to  only  applicable 

information,  i.e.,  no  historic  reports,  only  current  or  last  two 
or  last  hour. 

For  FT's,  forecasts  for  time  periods  past  should  be 
eliminated;  format  should  resemble  the  format  seen  in  WX 
office.  Normal  format. 

For  FD's,  screen  should  show  only  altitudes  I  request. 
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OVERALL  WEATHER  DESIGN 


The  overall  Weather  Data  Link  design  was  rated  FAIRLY  GOOD. 
Problem  areas  listed  were  centered  on  the  information 
presentation,  e.g.,  letter  size,  amount,  etc.  The  observed  level 
of  significance  was  less  than  0.05  suggesting  a  preferred  rating, 
i.e.,  "5,"  or  "FAIRLY  GOOD." 


Question  17.  Improper/ incomplete  system  designs  can  result  in 
pilot  errors.  Overall,  how  would  you  rate  this  Data  Link 


system  -  Weather  Services  design.  Use  the  standard  scale 
provided.  If  you  feel  design  problems  exist,  specify  the 
areas  you  find  unsatisfactory,  i.e.,  where  the  potential  for 
errors  is  most  prevalent.  Please  provide  your  suggestions 
on  how  we  might  redesign  the  system  to  overcome  these 
problems. 

System  Design  -  Weather  Rating 


GAT 

Avg  Std  A; 

5.38  0.95  5 

Problem  areas/ suggestions: 


B-727 


0.85 


Overall 
Ava  Std 

5.14  0.71 


#1  By  changing  the  size  of  the  letters.  Emphasis  can  be  made 
to  the  pertinent  information.  WX  information  for  the  most  part 
is  nonemergency  in  nature  and  can  be  requested  in  a  relatively 
relaxed  atmosphere. 

#2  Automatically  having  the  Data  Link  able  to  provide  6-9 
LOCID's  for  the  area  the  plane  is  flying  through. 

#3  Less  clutter,  screen  more  information;  probably  requiring 
sequential  screens. 

B-727  Pilots 

#2  If  visual  presentations  available  through  the  National  WX 
Service  could  be  displayed,  it  would  enhance  the  system  greatly. 

#3  Locate  the  device  in  the  cockpit  for  the  least  effort  for 
the  pilot  both  for  observation  and  operation.  Reduce  keystrokes 
to  a  minimum. 


Not  enough  information  available  in  current  system. 
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#5  Same  as  question  13. 


#6  I  think  critical  messages  such  as  altitude  should  be 
acknowledged  by  both  pilots  before  disappearing  from  screen.  I 
think  the  program  should  be  revised  to  include  the  appropriate 
words  of  "Climb"  or  "Descent,"  or  else  eliminate  those  words  and 
include  "Change  to" . . . 

SUGGESTED  SERVICES. 

OBSERVATIONS .  The  pilots  made  numerous  suggestions  on  weather 
services  to  be  part  of  future  Data  Link  designs.  A  few 
suggestions  were  as  follows:  SIGMETs,  radar  summaries, 
windshear,  hourly  sequence  reports,  etc.  One  pilot  wants 
to  be  able  to  show  weather  sequences  of  a  single  station  to 
determine  trends  (if  any) . 


Question  18.  The  design  of  a  major  system  normally  goes 
through  numerous  iterations.  What  weather  services  would  you 
consider  an  absolute  requirement  in  future  designs  and/or 
as  part  of  operational  systems? 

Suggested  services: 

GAT  Pilots 

#1  See  questions  10  and  12. 

#2  Adding  SIGMET's  to  the  current  four  chores. 

#3  Those  currently  used  with  more  information:  visibility, 
restrictions  to  vision,  runway  conditions,  icing  conditions, 
etc.  Ground  wx  radar  pictures  in  color. 

#4  Possibly  radar  summaries  for  precipitation. 

B-727  Pilots 

#1  Hourly  sequence  reports. 

#2  See  above  (question  17)  . 

#3  Windshear  info  —  This  should  be  sent  out  automatically. 

#4  Need  to  be  able  to  request  information  as  needed. 

#5  Same  as  question  #13. 

#6  (1)  Latest  hour  (elapsed  time)  of  WX  reports,  (2)  WX  at 

requested  locations. 
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#7  Runway  conditions.  Have  ability  to  show  several  WX 
sequences  of  same  station  to  pick  up  a  trend. 

#8  Any  segment  that  effects  any  area  near  our  flight  path 
should  automatically  be  sent  to  us.  Windshear,  any  unusual 
weather . 

UNWANTED  SERVICES. 

OBSERVATIONS .  The  number  of  weather  services  that  should  not  be 
part  of  Data  Link  were  minimal.  One  pilot  would  take  all 
services. 


Question  19.  What  weather  services,  if  any,  should  not  be  part 
of  the  Data  Link  system? 

£AT_JPiLots 

#2  Area  forecasts — too  much  information  to  try  to  put  on  little 
screen. 

B-727  Pilots 

#1  None;  I'll  take  all  I  can  get. 

#3  No  change  necessary. 

#4  Should  be  able  to  uplink  all  services  as  needed. 

#6  (1)  Alt  setting  in  millibars. 

(2)  Cloud  covering  in  1/10' s  of  sky  coverage. 

(3)  Types  of  clouds,  i.e.,  (as  AC)  other  than  CB. 

If  there  are  anv  other  comments  vou  would 
wish  to  express  on  the  Weather  portion  of 
the  Data  Link  system,  use  this  page  and  the  back. 

GAT  Pilots 

#4  Possibly  using  ground  radar  services  for  Data  Link  screens. 


B-727  Pilots 

#3  Condense  messages  as  much  as  possible. 
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OVERALL. 


OBSERVATIONS .  The  Main  Menu  was  rated  GOOD  on  all  factors  by 
GAT  pilots.  The  overall  appearance  was  rated  GOOD  by  B-727 
pilots;  however,  the  other  factors  were  only  rated  FAIRLY  GOOD. 
GAT  pilot  comments  were  concerns  with  the  Main  Menu  as  the 
number  of  services  available  grows  and  the  screen  color.  One 
B-727  pilot  emphasized  that  Data  Link  should  do  anything  that 
Voice  can - "keep  voice  as  a  backup." 


Question  20.  The  Data  Link  display  Main  Menu  provided  for 
selection  of  WX  services  and  message  retrieval.  Rate  the  Main 
Menu  on  the  following  factors.  Use  the  standard  scale  in 
making  responses  and  circle  the  appropriate  number.  Please 
comment . 


GAT 

B- 

•727* 

Overall 

Avq 

Std 

Avq 

Std 

Avq 

Std 

Overall  appearance 

6.0 

0.0 

6.14 

0.69 

6.09 

0.54 

Clutter 

6.0 

0.0 

5.14 

1.35 

5.45 

1.13 

Time  required  to  interpret 

6.0 

0.0 

5.86 

0.69 

5.91 

0.54 

Amount  of  information 

5.5 

1.0 

5.14 

1.68 

5.27 

1.42 

*Based  on  seven  responses 


Comments : 

GAT  Pilots 

#1  Obviously  (amount  of  information)  will  change  with  more 

services  provided. 

#2  Having  Main  Menu  show  location  of  last  request  of  each  type 
of  information  is  a  good  idea.  When  different  report  is 
requested,  most  recent  LOCID  should  automatically  appear  on  the 
request  screen,  to  be  overridden  by  either  a  new  LOCID  or 
Keyboard . 

#3  Would  probably  require  sequential  screens  to  provide 

additional  information. 

#4  Should  break  away  from  normal  green  format  to  a  less 
distracting  color. 

B-727  ..Pilots 

#3  Earlier  comments  regarding  the  display  should  be  noted. 

#4  Other  than  legibility  of  some  numbers,  display  looked  good. 
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#6  A  comment  on  this  critique  —  All  Data  Link  screen 
representations  should  be  included  in  this  critique  to  refresh 
our  memories  since  the  display  is  new  to  us. 

#7  No  need  to  show  ID  of  last  station  after  WX  information. 

#8  A  pilot  should  be  able  to  pull  up  anv  WX  information,  ATC 
requests,  100%  of  what  we  do  with  voice  —  Keep  voice  as  a 
backup. 

WORKLOAD. 

OBSERVATIONS .  The  ATC  workload  ratings  given  by  GAT  pilots 
(acceptable)  were  almost  a  full  point  higher  than  those  given  by 
B-727  pilots  (low  workload) .  This  may  be  due  to  the  fact  that 
the  GAT  pilot  was  alone  whereas  the  B-727  portion  utilized  a 
two  person  crew.  Ironically,  however,  the  GAT  pilots  rated  the 
workload  lower  for  the  weather  services.  The  standard 
deviation  with  B-727  pilots  was  slightly  higher.  The  binomial 
test  was  performed  on  this  data  by  forming  two  categories  (1, 
2,  3  -  LOW)  and  (4  and  5  -  HIGH)  on  the  assumption  that  a 
rating  1,  2,  or  3  was  Acceptable,  whereas  a  rating  of  4  or  5 
was  Unacceptable.  For  all  ATC  and  WX  services  the  p  values 
were  all  less  than  0.05,  except  terminal  forecasts  and 
PIREP's.  This  indicates  that  for  all  services,  except 
terminal  forecasts  and  PIREP's,  a  preferred  rating  category  was 
chosen.  This  category  was  the  low  (1-2-3)  category. 


Question  21.  For  each  of  the  services  (and  overall),  how  would 
you  rate  your  workload  as  it  appeared  during  the  flight.  Use 
the  scale  below.  Indicate  a  rating  for  each  and  make  comments 
where  applicable  (e.g.,  ratings  of  5  or  4) . 

5  Completely  saturated 

4  High  workload 

3  Acceptable  at  all  times,  not  too  high  OR  low 

2  Low  workload 

1  Very  low,  bored,  need  activity 


Workload  ratina  for: 

GAT 

1 

B- 

•7272 

Overall 

Altitude  assignment 

Ava 

Std 

Ava 

sta 

Avq 

Std 

3.33 

0.58 

2.43 

0.53 

2.7 

0.67 

Frequency  Change 

3.33 

0.58 

2.43 

0.53 

2.7 

0.67 

Surface  observations 

2.0 

1.0 

2.86 

3.14 

2.6 

0.84 

Terminal  forecast 

2 . 0 

0.0 

3.14 

1.07 

2.8 

1.03 

Pilot  report 

2 .33 

0.58 

3 . 0 

0.89 

2.78 

0.83 

Winds/temperature  aloft 

2.0 

0.0 

2.86 

0.69 

2.6 

0.70 

Overall 

3.0 

0.0 

2.75 

0.71 

2.82 

0.60 

^■Based  on  three  responses 

^ Based 

on  seven  responses 

Comments : 


GAT  Pilots 

#2  First  frequency  change  comes  just  a  level  off,  VOR  passage, 
trying  to  get  plane  settled  down  for  main  part  of  flight,  prefer 
Data  Link  at  this  point  so  I  can  respond  when  able.  Other 
frequency  changes  at  low  workloads,  welcomed  the  change. 

B-727  Pilots 

#1  On  UA. . . "Not  Observed.” 

#2  As  pilot  flying,  when  the  flying  workload  was  high,  the 
Data  Link  functions  increased  the  workload  further.  When 

workload  was  normal,  interface  with  Data  Link  was  good  (Normal 
and  acceptable) . 

#3  For  the  services  normally  required,  workload  rates  of  3. 
LOGIC  DESIGN  -  OVERALL. 

OBSERVATIONS .  The  LOCID  menu  layout  was  rated  FAIRLY  GOOD  by 
GAT  pilots  and  FAIR  by  B-727  pilots.  The  GAT  pilots  comments 
were  centered  on  real  world  implementation.  The  B-727 
pilots  requested  more  LOCID' s  and  a  keyboard.  The  observed 
level  of  significance  was  between  0.05  and  0.10,  which 
indicates  variety  of  pilot  rating. 


Question  22.  After  initiating  a  weather  service  request,  a  LOCID 
menu  appeared  for  you  to  input  the  location  of  interest.  Rate 
the  menu  layout  using  the  standard  scale  provided.  Please 
comment  and  provide  suggestions  for  improvement. 

GAT  B-727  Overall 

Avg  Std  Avg  Std  Avq  Std 

5.75  0.5  4.5  0.76  4.92  0.90 

Comments : 

SAT-PUsts 

#1  Good.  I'm  curious  as  to  how  the  LOCID' s  presented  will 
be  selected;  i.e.,  will  the  operator  program  six  LOCID's?  Will 
the  system  provide  LOCID's  close  to  your  route  of  flight? 

#2  If  software  would  permit,  LOCID's  that  do  not  have 
current  data,  or  none  available,  should  be  either  flagged  or 
eliminated  from  display. 

#3  Worked  well  in  this  exercise,  but  impractical  in  real  world. 
Need  means  for  pilot  to  input  LOCID's. 
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#4  I  think  individual  keyboard  input  for  identifier  would  be 
easier. 

B-727  Pilots 

#1  Given  the  display  as  it  is,  this  method  of  station  selection 
probably  can't  be  improved  upon  too  much. 

#3  Efforts  should  be  made  to  make  available  more  stations. 

#4  Will  comment  elsewhere  on  LOCID. 

#5  Same  as  question  13. 

#6  A  little  cluttered. 

#7  Use  keyboard  to  enter  ID.  Don't  need  additional  step  of 
hitting  "Send"  button.  Leave  word  "SENT"  on  screen. 

LOGIC  OPTIONS. 

OBSERVATIONS .  The  logic  flow  associated  with  the  LOCID 
menu/weather  retrieval  was  acceptable  by  all.  One  pilot  did 
not  want  to  have  to  touch  to  retrieve  the  weather  when 
available.  One  pilot  commented  that  since  the  "SENT"  message 
was  so  brief,  he  went  to  the  MAIN  menu  to  determine  if  the  MAIN 
menu  indicator  was  present. 


Question  23.  After  entering  a  weather  request,  the  system 
remained  in  the  LOCID  menu  (access  to  the  Main  Menu  is  possible 
by  touching  "Main"  at  lower  left) .  Upon  delivery  of  the 
requested  service  to  the  Data  Link  system,  a  message  appeared  at 
the  bottom  of  the  display  indicating  its  availability.  Upon 
touching  the  area,  the  weather  information  was  presented. 
Was  this  logic  flow  acceptable? 

GAT  B-727 

Yes  1,2,4  1,2, 3, 4, 5, 6, 7, 8, 

No 

Maybe  3 

Comments : 

GAT  Pilots 

#1  Once  again,  I  did  not  see  the  "SENT"  message  and, 
consequently,  went  to  the  MAIN  menu  to  look  for  the  "?." 

#2  Acceptable;  not  as  necessary  as  the  ATC  MSG  prompt. 
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#3  Do  not  need  touch  activity,  just  display  information  with 
audible  announcement. 

B— 727  Pilots 

#1  Whatever  is  there  will  stay  there  until  the  user  wants  it. 
A  good  feature. 

#2  Rather  than  (?) — if  available,  so  state.  If  you  don't, 

pilot  doesn't  know  if  it  is  a  malfunction  or  unavailable. 

#3  (didn't  answer,  but  said...).  For  weather  information, 
I  would  prefer  to  have  two  keystrokes:  (1)  Select  LOCID  and  (2) 
Select  Service. 

#4  It  follows  the  other  ?  (MAIN  menu)  —  No  time  was  lost. 

The  ability  to  make  multiple  weather  service  requests  while 
waiting  for  message  delivery  was  considered  a  necessary  option. 
One  pilot  did  state  that  "Data  Link  should  answer  one  reguest  at 
a  time...."  The  words  "desirable,"  "definitely,"  and  "time 
efficient"  were  used  in  the  comments. 


Question  24.  After  making  a  weather  request  and  while  waiting 
for  the  information  to  appear,  you  had  the  ability  to  make 
another  weather  request  (or  view  message  lists) .  This  capability 
was  not  exercised.  Do  you  think  that  this  is  a  necessary  option 
in  the  Data  Link  system? 

GAT 

Yes  1,2,3 

No  4 

Maybe 

Comments : 

GAT  Pilots 

#2  This  was  not  available  for  today's  simulation.  On  Winds 
Aloft  and  Terminal  Forecasts,  pilot  may  want  to  view  several 
stations. 

#3  This  attempt  crashed  system  during  my  exercise. 

#4  I  think  the  Data  Link  should  answer  one  request  at  a 
time;  however,  request  of  several  different  locations  during  one 
flight  is  essential. 


B-727 

1,2, 3, 4, 5, 6, 7, 8 
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B-727  Pilots 


#1  The  whole  idea  should  be  to  get  as  little  or  as  much 
information  quickly  or  as  leisurely  as  one  wishes  commensurate 
with  his  existing  workload. 

#2  Desirable.  More  time  efficient. 

#4  Definitely,  but  not  necessarily  through  the  use  of  the 

keyboard  device. 

VISUAL  ANNUNCIATION. 

OBSERVATIONS .  The  annunciation  (with  "Touch  for  ATC 
Message")  logic  was  acceptable.  Emergency  messages  should 
be  made  by  voice.  Automatic  delivery  of  ATC  messages  was 
not  a  desired  feature.  Pilot  action  to  retrieve  the 
information  is  required.  Different  tones  or  number  of  tones 
was  suggested  to  discriminate  weather  and  ATC  messages.  The 
test  statistic/p  value  indicates  a  preferred  choice,  i.e., 
YES,  in  parts  A  and  B. 


Question  25A.  An  incoming  ATC  message  was  annunciated  by  "Touch 
For  ATC  Message"  at  the  bottom  of  the  screen.  Is  this 

operation  logic  acceptable  in  all  situations? 

GAT  B-727 

Yes  1,2, 3, 4  1,2, 3, 4, 5, 6, 7, 8 

No 

Comments : 

GAT  Pilots 

#1  (Emergency/urgent)  messages  need  to  be  sent  by  voice. 

B-727  Pilots 

#1  Urgent  ATC  messages  should  be  accomplished  by  voice. 

Question  25B.  Or,  for  example,  in  a  non-busy  situation,  such  as 
when  in  the  MAIN  menu  and  no  weather  messages  pending:  Should  the 
ATC  message  automatically  appear  without  being  required  to  touch 
the  screen?  Comment  on  other  situations  of  interest. 


GAT 

B-727 

Yes 

4 

No 

3,4 

1,2, 3, 5, 6, 7, 8 
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Comments: 


GAT  Pilots 

#1  That  would  be  a  nice  feature. 

#2  Having  the  pilot  touch  the  screen  for  ATC  MSG  is  good. 
Alerts  pilot  that  something  important  may  be  coming  in.  There 
should  be  different  tones  or  number  of  tones  for  ATC  vs  WX 
messages. 

#4  Some  method  must  be  used  to  let  ATC  know  that  the  message 
was  received. 

B-727  Pilots 

#1  See  previous  comment. 

#2  Also — suggest  flashing  the  annunciator  so  that  message  will 

not  be  overlooked.  Stop  flashing  when  acted  upon. 

#3  ATC  messages  should  be  positively  acknowledged. 

#4  I  believe  my  comment  elsewhere  was,  "I'll  accept  the 
messages  when  ready." 

#6  The  screen  could  "default"  to  the  latest  ALT  heading,  freq, 
route ,  etc . 

#8  Pilot  action  should  be  required. 

The  Chi-square  test  resulted  in  significance  levels  as  follows 
for  the  difference  between  Yes  and  No  answers  on  questions  25A 
and  25B. 

25A  Chi-square  =12.0  p  <  .01 
25B  Chi-square  =4.45  p  <  .05 
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MESSAGE  LIST-LOGIC  OPTIONS. 


OBSERVATIONS .  The  ability  to  retrieve  a  list  of  messages 
was  a  desirable  option.  One  pilot  would  prefer  a  printer  in 
addition  to  the  display.  The  order  of  messages  was  a  problem 
area  for  the  tested  system.  Old  information  (frequencies, 
altitudes)  are  not  necessary;  present  only  the  most  recent 
message  for  each  service.  The  test  statistic/p  value  confirms 
that  a  preferred  response  was  made,  i.e.,  YES. 


Question  26.  Do  you  think  the  ability  to  retrieve  a  list  of 
messages  received  (from  the  MAIN  menu)  is  a  desirable  option  on 
the  Data  Link  system? 

GAT 

Yes  1,2, 3,4 
No 


B-727 

1,2, 4,5, 6, 7, 8 


Comments: 

GAT  Pilots 

#2  Most  recent  message  should  be  on  top  of  screen,  scrolling 
off  bottom.  Not  the  page  wrap  that  is  currently  programmed. 

#3  Very. 

#4  However,  again  I  think  buttons  should  be  used  instead  of 
touch  pad. 

B-727  Pilots 

#2  Problem,  after  eight  messages  you  can  start  losing  them 
due  to  overflow. 

#3  A  better  method  would  be  to  have  messages  come  out  on  a 
printer  along  with  the  usual  presentation. 

#4  Definitely — Important — Do  not  delete. 

#6  However,  you  can  rid  it  of  previous  frequencies  when  there 
have  been  two  changes,  i.e.,  the  frequency  I  was  on  three 
frequencies’  ago  is  useless  to  me  and  just  is  clutter,  so 
dropping  the  second  historic  frequency  in  order  to  the  clear  the 
screen  up  would  be  helpful. 

#7  They  must  be  in  chronological  order.  Recommend  most  recent 
on  bottom.  No  need  to  retain  more  than  last  frequency 

assigned  and  last  altitude  assigned. 
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The  Chi-square  test  showed  a  significance  level  as  follows  for 
the  difference  between  Yes  and  No  answers  in  question  26. 

Chi-Square  =8.33  p  <  . 01 

SCENARIO  SUTIABILITY . 

OBSERVATIONS .  The  scenarios  were  appropriate  for  evaluating 
Data  Link.  Most  would  have  preferred  longer  runs  and/or 
additional  runs.  One  pilot  said,  "Work  me  harder  next  time." 
The  test  statistic/p  value  confirms  that  the  YES  was  a 
preferred  rating. 


Question  27.  Do  you  think  the  scenario  you  flew  in  for  this 
test  was  appropriate  for  evaluating  Data  Link?  Please  comment. 


GAT 

Yes  1,2, 3,4 

No 


B-727 

1,3, 4, 5, 6, 7, 8 
2 


Comments : 

GAT  Pilots 

#1  Flying  both  scenarios  twice,  one  in  voice  and  one  in  Data 
Link,  would  have  been  nice. 

#2  Although  route  was  relatively  short,  after  initial  climb  and 
level  off,  most  IFR  flights  are  low  in  workload,  exception  being 
turbulence,  or  possible  East  Coast  plan. 

#3  Sort  of.  A  longer  trip,  utilizing  more  realistic  terminal 
operations  probably  would  be  more  revealing. 

#4  Especially  liked  weather  reports  on  ground. 

B-727  Pilots 

#1  Runs  could  have  been  a  little  longer. 

#2  I  think  longer  legs  would  be  advisable  to  give  the  pilot 
flying  longer  to  interface  with  the  Data  Link. 

#3  It  was  adequate  for  this  phase  of  development. 

#4  It  was  busy  enough  to  exercise  the  crew  and  Data  Link. 

#8  Work  me  harder  next  time. 
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The  Chi-square  test  showed  a  significance  level  as  follows  for 
the  difference  between  Yes  and  No  answers  in  question  27. 

Chi-square  =8.33  p  <  .01 
DISPL7 Y  LOCATION. 

OBSERVATIONS .  The  cockpit  location  of  the  Data  Link  display 
was  acceptable.  The  B-727  pilots  emphasized  the  lack  of  "real 
estate"  in  the  727;  the  weather  radar  position  for  Data  Link  was 
acceptable,  but  the  weather  radar  is  needed  as  well.  The 
test  statistical  p  value  confirms  that  the  YES  was  a  preferred 
rating. 


Question  28.  Was  the  cockpit  location  of  the  display  acceptable 
given  the  type/ importance  of  the  information? 


GAT 

Yes  1,2, 3, 4 
No 


B-727 

2, 3, 4, 5, 6, 7, 8 

1 


Comments : 

GAT  Pilots 

#1  Combining  into  an  EFIS/DFIS  would  be  even  better;  but  with 
conventional  instruments  it  was  fine. 

#2  Location  was  easily  visible  from  either  seat,  out  of  path  of 
normal  scan,  but  near  enough  for  a  flashing  message  to  be 
caught  by  eye 

B-727  Pilots 

* 

#1  A  727  cockpit  is  a  difficult  place  to  locate  this  device. 

Maybe  dua]  heads  closer  to  users.  * 

#2  Better — Yoke  mounted  dual  displays  to  give  better  "head  up" 

capability 

#3  For  a  multiple  crew  operation,  the  location  was  adequate. 

For  a  single  pilot  operation,  the  location  should  be  more 
accessible. 

#4  But  in  the  future  it  should  be  on  a  CRT  elsewhere.  I  need 
the  WX  radar  more. 

#8  But  where  do  we  put  the  radar? 
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The  Chi-square  test  showed  a  significance  level  as  follows  for 
the  difference  between  Yes  and  No  answers  in  question  28. 


Chi-square  =8.33  p  <  .01 
INPUT  TECHNOLOGY. 

OBSERVATIONS .  The  touch-sensitive  input  technology  for 
airborne  use  was  acceptable  to  most.  Concerns  were 
expressed  about  use  in  turbulence.  Two  pilots  found  the 
technology  unacceptable  and  suggested  buttons  or  a  mouse.  One 
pilot  called  the  technology  "a  must.” 


Question  29.  What  do  you  think  of  the  touch  sensitive 
input  technology  as  a  concept  for  airborne  use?  Please  comment. 


GAT 

Acceptable  1,2,3 

No  preference 
Unacceptable  4 

Comments : 


B-727 

1,2, 5, 6, 7, 8 

3 

4 


GAT  Pilots 

#2  May  be  difficult  for  people  with  FAT  fingers,  or  in 
turbulence.  Did  not  always  pick  up  on  first  or  second  time. 

#4  Buttons  or  something  else.  Even  in  this  scenario  wrong  pads 
were  touched  by  other  fingers  and,  if  accidentally  touched  twice, 
the  screen  changed  each  time. 

B-727  Pilots 

#3  The  system  is  open  to  some  error.  At  times  the  requests  did 
not  take.  In  turbulence,  it  could  be  difficult.  It's  not  as 
positive  as  a  button. 

#4  I  would  rather  see  a  mouse  and  enter  button. 

#6  A  must. 

#7  This  system  worked  fine. 
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PRE-TEST  TRAINING. 


OBSERVATIONS .  Most  everyone  was  satisfied  with  the  briefing 
and  training  received  on  Data  Link.  One  pilot  would  have 
preferred  more  information  to  study.  The  test  statistic/p 
value  confirms  the  Yes  response  preference. 


Question  30.  Were  you  satisfied  with  the  briefing/training 
you  received  on  the  Data  Link  system?  Please  comment. 

GAT  B-727 

Yes  1,2,4  1,2, 3, 4, 5, 6, 7, 8 

No  3 

Comments : 

GAT  Pilots 
#1  Good  job. 

#2  Understood  system  and  ease  of  use.  Most  is  straightforward 
logic.  Very  acceptable. 

#3  Would  have  preferred  more  information  to  study. 

B-727  Pilots 

#3  The  briefing  was  thorough  and  made  the  test  easy  and 
effective. 

#4  Very  good  and  efficient. 

The  Chi-square  test  showed  a  significance  level  as  follows  for 
the  difference  between  Yes  and  No  answers  in  question  30. 

Chi-square  =8.33  p  <  .01 
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